anon_mibo said in #4003 6d ago:
With Facebook apparently making multiple $100M cash buyouts of individual AI researchers, billions and billions of investment dollars pouring in to AI related industry, and a general atmosphere of extreme hype, one starts to wonder where the matching profit is going to come from. Oh sure if you spun up an actually fully capable PhD equivalent remote-worker AI employee tomorrow you might be able to charge thousands of dollars a month and sell subscriptions by the millions. But how close are we to this actually? And what happens if it doesn't materialize as quickly as the industry is betting on, or the economics are no good?
My general impression of LLM-based AI has been somewhat disappointing. They are certainly useful for a lot of things, but not *that* useful. I pay $20/month. I wouldn't pay $200, and not because I don't have a lot of work to do. My friends who run more serious companies betting on this stuff have started quietly saying it's just not there yet, that they can't maintain coherence and reasoning quality outside their trained domain. An investor reports to me that the best people are reporting about 20% speedup. Very smart programmers are making a lot of noise challenging people to show actually impressive results seriously accelerated with LLMs, but it's unclear to me if there's a response. There have been a few impressive stunts, but I've not seen anything that convinces me it's more than a fractional speedup for already existing talent and teams, let alone a full on replacement. YCombinator seems to be all in, but is it delivering?
Then we have the technical angle. Token prediction in principle could mean a full world simulation that understands everything and is fully intelligent. Or it could mean a glorified markov chain. Which is closer to reality? The transformer model feels architecturally much more like a glorified markov chain with some tricks to fit an interpolation curve instead of a lookup table. Impressive in its emergent capabilities, but also noticeably lacking. Can it be patched with reinforcement learning, "reasoning tokens", data augemntation? Somewhat. But just patched. The architecture is still effectively a sort of markov chain. It captures some regularities but not all. It's just not the right *shape* for AGI. The lack of ability to operate outside of its interpolative trained domain seems like a permanent cap on the autonomy of LLM agents, relegating them to highly-managed junior roles, not trusty expert employee-agents.
Meanwhile the reveue growth of Anthropic and OpenAI is pretty impressive. The main show has been code generation, menial data munging, entertainment, and spam spam spam. Someone must actually be figuring out how to use these things, but I hear the economics are still negative. The companies are betting on durable monopoly positions and the ability to back off the capex, and increase margins. But token interpolation seems like a commodity service with low switching costs, constant threat of disruption, and low margins. The most durable business here is the low-IQ entertainment companion that develops a codependent relationship with the user. LLM psychosis and Grok's robo-waifu are early indicators here. But if *that's* the business case and not transformative AGI, a lot of the investment thesis collapses.
There's a lot of value here don't get me wrong, but the economics are unclear and I can't shake the feeling that the investor hype has been based on a speculative scifi narrative that the technology isn't ready to live up to. It's reminiscent of previous tech bubble crashes. The markets are insane and often counter-fundamentals, and most of the big labs just got fat $200M contracts from the military which will help smooth things out, but even the fact that they sought that out is suggestive of the underlying situation.
What do you guys think? Is it time to short AI, or is this just a slight moment of uncertainty on the straight road to singularity?
My general impression of LLM-based AI has been somewhat disappointing. They are certainly useful for a lot of things, but not *that* useful. I pay $20/month. I wouldn't pay $200, and not because I don't have a lot of work to do. My friends who run more serious companies betting on this stuff have started quietly saying it's just not there yet, that they can't maintain coherence and reasoning quality outside their trained domain. An investor reports to me that the best people are reporting about 20% speedup. Very smart programmers are making a lot of noise challenging people to show actually impressive results seriously accelerated with LLMs, but it's unclear to me if there's a response. There have been a few impressive stunts, but I've not seen anything that convinces me it's more than a fractional speedup for already existing talent and teams, let alone a full on replacement. YCombinator seems to be all in, but is it delivering?
Then we have the technical angle. Token prediction in principle could mean a full world simulation that understands everything and is fully intelligent. Or it could mean a glorified markov chain. Which is closer to reality? The transformer model feels architecturally much more like a glorified markov chain with some tricks to fit an interpolation curve instead of a lookup table. Impressive in its emergent capabilities, but also noticeably lacking. Can it be patched with reinforcement learning, "reasoning tokens", data augemntation? Somewhat. But just patched. The architecture is still effectively a sort of markov chain. It captures some regularities but not all. It's just not the right *shape* for AGI. The lack of ability to operate outside of its interpolative trained domain seems like a permanent cap on the autonomy of LLM agents, relegating them to highly-managed junior roles, not trusty expert employee-agents.
Meanwhile the reveue growth of Anthropic and OpenAI is pretty impressive. The main show has been code generation, menial data munging, entertainment, and spam spam spam. Someone must actually be figuring out how to use these things, but I hear the economics are still negative. The companies are betting on durable monopoly positions and the ability to back off the capex, and increase margins. But token interpolation seems like a commodity service with low switching costs, constant threat of disruption, and low margins. The most durable business here is the low-IQ entertainment companion that develops a codependent relationship with the user. LLM psychosis and Grok's robo-waifu are early indicators here. But if *that's* the business case and not transformative AGI, a lot of the investment thesis collapses.
There's a lot of value here don't get me wrong, but the economics are unclear and I can't shake the feeling that the investor hype has been based on a speculative scifi narrative that the technology isn't ready to live up to. It's reminiscent of previous tech bubble crashes. The markets are insane and often counter-fundamentals, and most of the big labs just got fat $200M contracts from the military which will help smooth things out, but even the fact that they sought that out is suggestive of the underlying situation.
What do you guys think? Is it time to short AI, or is this just a slight moment of uncertainty on the straight road to singularity?
referenced by: >>4014 >>4077
With Facebook appare