Question: How bad would the whole AI/search/chat situation have to get — how much real-world harm have to be done — before any of the tech companies pulled their version from the market? 

Answer: The publicly-held companies might pull theirs in response to a stock-market collapse, but the privately-held ones? I can’t imagine any circumstances short of legislative action that would cause them to pull back. They believe in the “move fast and break things” mantra, they think no publicity is bad publicity, and their technological justification is that the bots will improve only through iteration. 

UPDATE: So Microsoft — one of the public companies in this racket — hasn’t taken down Bing Chat but has “lobotomized” it. Sydney, we hardly knew ye.