Your all in one solution was a marketing hoax that killed AI for several decades.
Interesting, I always blamed this on DARPA cuts, and The Fifth Generation Computing Project. I don't really know enough about the history of this truthfully, you might be right I don't know.
LISP machines have never been cheaper or more readily available than right now due to FPGAs. Still no interest.
I think a lot of this has to do with momentum. Ever since the IBM System/360 people have been far from eager to start from ground zero. There's a lot of risk, costs, and uncertain rewards (especially in terms of capital). It doesn't help that of course without economies of scale you'd never actually be able to do more computations on a Lisp machine in terms of raw flops or really any other relevant measure. Further FPGA's don't give you tagged memory, which is still an important aspect of a performant Lisp system and historically one of the most costly components.
"The right thing" is a big system dependent on all its parts being right - its marketed as fully solving all problems it was designed to do, a LISP machine e.g. solved all problems with LISP memory use by tagging objects in hardware.
Yah, tagged memory and garbage collection in hardware were pretty sweet for Lisp. This is sort of the nature of the game though, I doubt C would be excellent on many non-von Neumann architectures for example. Regardless it seems to me that this is a crucial point that we should learn from in the future. We should certainly look to make compromises with reality as it exists to make the best product for us rather than some ideal essence of perfect computation. "The Right Thing" in the context of a different reality is irrelevant, if not harmful.
"Worse is better" system produces systems that are better in some critical aspects that customers demand and ignore everything else, lowering cost of production.
I'm not sure if this actually reflects the history of UNIX. Research UNIX was licensed for existing affordable hardware such as the PDP-7, PDP-11, etc. that were present in universities at the time, for exceedingly low prices. It then gained momentum as students wanted to continue to use the thing that was familiar to them from university, and because it turns out that UNIX is easy to port. It seems to me from the beginning cost and momentum were the motivators, along with ease of ports. I'm actually not trying to make a moral judgement here, evolution is not something you debate, to make moral claims about this would be like being fussed about not being able to grow new arms like Chameleons or something.
(I was going to go to bed but I can't sleep)