I didn&#x27;t spend that much, only $6500 AUD for a GB10 based Asus GX10 which is even slower than OPs, but I spent that because it makes for a great learning platform. Theres not much else that lets me fiddle with 128GB of RAM for my graphics processor, and it&#x27;s quite lovely to be able to run things as long as I like without worrying about my cloud instance being shut down.It&#x27;s not financially a good idea: renting really does beat owning, and cloud beats both if you&#x27;re only running inference on these machines. But I&#x27;m not just doing inference, and as a thing I can do silly stuff on to learn, it&#x27;s hard to beat!

The cost is obviously not that big of factor for OP as it might be for others. It&#x27;s actually refreshing to hear the candid viewpoint that he expresses here.

Privacy and offline operation are valuable or non-negotiable in some cases, but the difference is pretty categorical between what can run on a single card and what can run on a DGX GB200 NVL72 cabinet. Doesn&#x27;t mean it&#x27;s not worth seeing how far local models can be pushed. Not every problem needs a senior engineer.

It&#x27;s just a project I&#x27;m working on. I&#x27;m working on projects where AIs are processing and classifying large amounts of data that would be a lot of work for humans to do.

One year ago finetuned local LLMs had a significant edge over ChatGPT or Claude. Look up in YouTube all the DIY videos testing LLMs on their own machines with different setups.Remember: one year showed up to be a gigantic leap in regards to quality of results and innovation in the AI space. Agents weren&#x27;t really a thing and vibe coding wasn&#x27;t even invented as a term because the top notch tools at the time were lousy, with lovable being the frontrunner with its - in my view - sorry Tailwind recombination tool shaming AI to do the work.Then fall hit 2025 hit us, new year&#x27;s eve and suddenly there was such a massive surge of innovation and competition with ChatGPT Codex suddenly showing up.Remember: one year ago many now commonly used tools weren&#x27;t yet available like Nano Banana or Codex.&quot;The 25k are so vast&quot; - Yes, and no. For example, if the machine is bought for business usage I can deduct the costs from taxes. This roughly amount for 50% of the financial burden.So I jokingly use to say, that I pay only half the price for my Apple business machines. And yes, I am strict in this regard. Business means business. No private emails etc. nothing on my company computers.Maybe there are other options as well to reduce the financial expenses the dude mentions, but it doesn&#x27;t seem so.I would also go for leasing, this way already the monthly payments can be deduced and I don&#x27;t need to buy and maybe resell the machine.Apple is a luxury good. Without business usage or at least partly using it for business as well as private (mixed usage in tax reports) I wouldn&#x27;t buy the devices or think twice.Apple under Cook evolved into a Gucci like luxury brand, that is more and more a rip off than quality delivered, especially considering the latest OS updates for Mac, iOS and iPad. Apple is a mess, following Microsoft Windows&#x27; footsteps happily, because the CEO is as has been correctly assessed, no product guy.But I stop with my rant here.Always try to use tax deduction as leverage for your computer expenses. Every citizen should invest in basic knowledge about that.Even a 10-20% professional usage for work (mixed usage) gives you a noticeable advantage over normal pay.

That hardware is costing him ~1$&#x2F;hour over 3 years. Presumably having it answer math questions was a tiny fraction of what he was using it for.

I’ve spent twice that on hosting movies and tv for Plex, so… I think they are worthy of my praise. What a healthy outlet for money.

I’m not usually one to ask this because learning to do a thing can be fun, but why exactly have you spent 25 thousand dollars on getting an LLM someone else made to answer maths exam questions?

I have three m3 512gb units and want a fourth to run an exo set up. Like you, I am worried about scammers. Let’s discuss if you still want to sell.<a href="https:&#x2F;&#x2F;calendly.com&#x2F;ryanwmartin&#x2F;open-office-hours" rel="nofollow">https:&#x2F;&#x2F;calendly.com&#x2F;ryanwmartin&#x2F;open-office-hours</a>

&gt;&gt; find a reliable place to sell it without getting ripped off by scammers.This is a real problem and why I&#x27;ve just about given up on ebay or fb marketplace, esp for computers. If you are in Canada though sellit9.com is a great solution to having to deal with sketchy buyers.

Something is very wrong in some countries if you have to get police protection to sell a f* computer. I get it’s on the expensive side but still….

If you&#x27;re in a decent sized city, you should be able to find a local buyer on Craigslist or FB Marketplace... Beyond that, for higher value, smaller items like your M3 Ultra, I would talk to your local police department and&#x2F;or library to see if you can do the exchange there. Larger libraries usually have a police officer on site or nearby, and the PD office near you may also provide a &quot;safe&quot; exchange location... I&#x27;d bring a monitor&#x2F;keyboard&#x2F;mouse so you can demonstrate the system working properly.YMMV but between your nearest PD office and Library, you should be able to use one or the other for your exchange of goods&#x2F;money. The biggest thing I&#x27;ve sold is a mid-range video card during late covid (I managed to get a better one via newegg shuffle) so I sold the old one (RX 5700XT -&gt; RTX 2080) to make up the difference a bit. I just did the exchange at the Starbucks near me for that.

You can fit Deepseek 4 Flash on two with TP 2 and 6 different streams at 65k context. 150 tok&#x2F;s

A pair of RTX6000 cards will give you a good performance boost due to tensor parallelism, though. I haven&#x27;t tried the newest predictive quants but I see about 35 tps when running the 8-bit Qwen 3.6 27B model on one board and about 50 tps on two. Probably could come close to 100 tps on an optimized setup with the latest GGUFs.Also, the 4-bit quants of MiniMax 2.7 will run at 100 tps or so with two cards, which is pretty decent. It doesn&#x27;t go any faster at all with 4 GPUs from what I&#x27;ve seen, so if you don&#x27;t actively need 384 GB of VRAM, 2x RTX6000 is a good place to be.

I got an RTX 6000 pro too. I like running locally, I&#x27;ve learned a lot more than if I had used an API and there&#x27;s less worry about overspending tokens. I accidentally spent $100 on claude api in like 2 days because I didn&#x27;t know what I was doing.The problem is that while one these gpus is a huge improvement over a laptop or a single 3090, you very quickly wish you had more. I would buy a second one, but I did the math and realized that with the current crop of models, 2 Blackwells doesn&#x27;t buy me any new capability that I didn&#x27;t have with one. So I would need a 3rd one. And when I buy a 3rd one I will feel like I want to running a higher quant, so then I will want a 4th.

Bizarrely, I brought a GPU new in Jun 2024, and there are sold ebay listings saying the used GPU is worth 4% more today.Of course, this is an unusual state of affairs; I see my GPU purchase as consumption, not investment.

&gt; I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment.It&#x27;s going to be a non-trivial haircut. This stuff depreciates pretty fast.

RTX 6000 is some-what obviously my fastest card but my biggest problem with the RT 6000 is the immense heat. The GPU itself is almost 200F and the exhaust from the fans itself is over 150F. I&#x27;m worried that my hard drives are going to fail. I was told that the GDDR7 is even hotter than the GPU which is surprising to me.After my last run, I&#x27;m going to wait for the new case I ordered to come in and cannibalize my kid&#x27;s PC that we built beginning of this year to form an entirely separate computer. And then figure out better ways to deal with the heat, especially with summer coming up. I&#x27;ll have to play around with undervolting and running vents directly outside my house to see if that helps.

Which of these has been the most productive for you? Sounds like you&#x27;ve enjoyed the RTX6000 the most?

You definitely want to get rid of your M3 Ultra before the M5 Ultra get officially announced.

I&#x27;ve seen a lot of sales on eBay for over $20k, but I don&#x27;t know if I believe it. Plus the lack of seller protection and the prevalence of scams on eBay make me too hesitant to actually want to risk it so I don&#x27;t know what to do haha

All of these have appreciated in value. How much are you looking for the Ultra?

A M3 ultra mac Studio can run models that do not fit in similarly priced computers with multiple Nvidia GPUs. And it will use a lot less electricity while still having good enough performance. Except the pre-filing perfs that are quite poor on the M3.

M5 pro 48GB should be good and future proof

Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.An RTX 6000 pro Blackwell is a pretty good card

If you are in the bay area, i&#x27;m happy to buy that M3 Ultra from you, i&#x27;ve been unsuccessfully looking for one and can&#x27;t find any.

No harm in listing it for $20k, and if it sells, that&#x27;s an easy $5-10k for you.

Well if it makes you feel better those frontier LLMs are all technically taking a big loss, and they may all be in your shoes after a few years.

I&#x27;m keeping that one for sure, I love it!

I&#x27;ll buy your macbook if you&#x27;re trying to get rid of it!

Yes the prices on eBay for the Mac Studio are all over the place, but I&#x27;ve seen sales for over $20k. I don&#x27;t know if I believe it but there&#x27;s enough to make me think if I can sell it for that price it would be worth it, but eBay has basically no seller protection so I&#x27;m not willing to take that chance.

You&#x27;ll probably make a profit by selling them today. I bought a M1 Max Studio with 64 GB last year off FB Marketplace for $1000 and today I&#x27;m seeing numerous 32 GB M1 Maxes for $1200-1500.

This is on a separate Windows PC, I don&#x27;t have it integrated with the Macs.

How do you use the RTX 6000 with the Macs? Exo? I would think that would be pretty snappy if configured properly.

I agree that all these datacenter companies like Coreweave are investing billions in technology that has a very fast depreciation curve and I don&#x27;t know how they will sustain income. The same goes for datacenters in space, what happens when those chips are obsolete? Will they sent astronauts to replace them or will they let them burn up and send new ones into orbit every year?I feel that the open weight models pale in comparison to the frontier models, and I believe that if the gap closes quickly, that the open weight vendors will stop releasing it for free.

I looked into the M3 Ultra 512GB Mac Studio before it was discontinued and the as best as I could determine it just wasn&#x27;t worth it... yet. The GFLOPS and memory bandwidth just arne&#x27;t there even though it can hold a much larger model in memory.But the trend here is interesting. I think by 2030 you&#x27;ll be able to buy fairly cheap hardware that is currently $10k+. I don&#x27;t know what this does to the trillions invested in AI data centers because the next NVidia architecture after Blackwell will essentially half the value of purchased cards overnight.I&#x27;m not convinced Apple has yet pivoted the Mac Studio line towards this market and the expected M5 Ultras in Q3 2026 will likely be an incremental improvement rather than big leap forward but I&#x27;d like to be proven wrong.

In the last year, I have bought an M3 Ultra Mac Studio with 512 GB, a Macbook Pro M5 MAX with 128 GB and an RTX 6000 Pro. I have spent around $25k so far, not including electricity. I figured worst case scenario I can sell them in the next year and only take a haircut as opposed to losing my entire investment.In comparison to just spending for tokens, the tokens would have been much cheaper and much much faster. I&#x27;ve been running against Gemma4:31b, Qwen3.5 and 3.6, and getting local LLMs to solve AMC 8&#x2F;10 math questions and it&#x27;s about 10-100x slower than just doing it online. When I tried it with ChatGPT late last year, it took about one night and $25 to solve about 1000 questions. Using my RTX 6000 and M3 Ultra and Gemma4:31b on both, it answered about 40 questions in 7 hours and I haven&#x27;t checked how good the answer is yet. At 800 watts (600 for RTX and 200 for M3 Ultra) and running for 7 hours, it solved around 40 questions.At the very least I&#x27;m going to try to sell my M3 Ultra if I can find a reliable place to sell it without getting ripped off by scammers.

I have a 5090 machine sitting idle that I&#x27;m considering turning into a machine for my own small team (3 devs).Are you willing to share any lessons learned, etc. that I could make use of? We are evaluating paying for a SOTA sub or trying this, and the talk about Qwen3.6-27B makes me want to try deploying this machine.

Wouldn&#x27;t that be a fairly ideal setup for layer parallelism? That doesn&#x27;t need the high-performance communication of tensor parallelism, and the high-concurrency regime would make it easy to keep the pipeline full with microbatches. You&#x27;d also be able to scale out your KV cache storage since that naturally splits layer-wise.

Does anyone here have experience running large models in a multi-GPU setup with several RTX 6000s in a high-concurrency regime and with large context lengths? (something like Deepseek 4 Flash, Minimax 2.7 etc.)For what it&#x27;s worth, I&#x27;ve been seeing ~100 tps with 4-bit MiniMax 2.7 on two RTX 6000 boards, just running under llama-server without any optimization effort at all. I have no serious long-context experience with that setup, but at 30K context it&#x27;s still above 90 tps.If you are happy with Qwen 3.6 27B, I would personally switch the 5090 out for 2x RTX 6000s and keep running 27B. That will give you ~2x your current throughput with a lot more headroom for multiple users. More important, it would buy time to see how things develop over the next few months before you spend a whole lot of money.

I administer a simple AI server in the office, which just uses a single RTX 5090 but is able to serve ~80 people throughout the day. I&#x27;m impressed by Qwen3.6-27b&#x27;s capabilities in agentic coding&#x2F;tasks so far. Devs say it&#x27;s not much different from Sonnet 4.6 on many tasks (sometimes it even outperformed it), 40-60 tok&#x2F;sec, up to 260k context. The server cost about $10k with all the bells and whistles.I spent a lot of time researching&#x2F;adding&#x2F;benchmarking many custom modifications to the software stack and its settings to make the server optimally handle the load with just 1 RTX 5090 without losing quality, but it&#x27;s still not enough, and the wait times in the queue are getting longer. We&#x27;re at the limits of the hardware, and I&#x27;m out of tricks.The experiment was kind of a success, and the CTO agrees we should scale it. With our own infra, we could run agents 24&#x2F;7 on everything. Currently, a lot of use cases for the cloud providers are completely blocked by PII&#x2F;trade secret concerns (our infosec department doesn&#x27;t buy the &quot;zero retention&quot; promise), plus you don&#x27;t have to think about billing&#x2F;budgets&#x2F;etc. anymore.Now I can&#x27;t decide how to scale it. On one hand, I&#x27;d like to run larger models. And we have the budget to buy, say, 8xH200. But in many benchmarks, the larger models that do fit in 8xH200 comfortably and can serve many parallel requests with acceptable speed&#x2F;quality don&#x27;t seem to outperform Qwen3.6 that much in agentic coding&#x2F;tasks to justify the price.So another option is just to buy a bunch of RTX 6000s and scale horizontally instead: run a copy of a midrange LLM like Qwen3.6 on each GPU. It&#x27;s cheaper and easier to scale&#x2F;replace, but then we&#x27;ll run into problems running larger models in the future if we have to, because of no NVLink support (say, if Alibaba &amp; Co. stop releasing ~30b models and&#x2F;or ~30b models start falling behind 400b+ models considerably)Does anyone here have experience running large models in a multi-GPU setup with several RTX 6000s in a high-concurrency regime and with large context lengths? (something like Deepseek 4 Flash, Minimax 2.7 etc.)

This is interesting but I am unsure how you make money out of this home setup, I would imagine if one would be offering consultancy to a business the business would make their own equipment&#x2F;infrastructure available, which would also give a better control of their data. But perhaps I am thinking this because I am thinking about very big companies. Then, on very small business I don’t see they having the use case with the budget to match the need. So is this for specific services for medium sized businesses? Can you explain this a bit?

Great article. I&#x27;m about to embark on a similar journey.... Doing a ton of AI development right now. Don&#x27;t need a server, but a very, very high end workstation is super appealing to me right now. Looking at $50-$80k. 1TB RAM. 2x RTX Pro 6000s. 64 core Threadripper Pro. As many 4tb or 8tb nvme drives as I can stuff.I envision NixOS at the core... then everything I need virtualized on top with KVM&#x2F;QEMU. Maybe a dual boot setup with Windows for gaming and Flight Simulator (but I could virtualize that too with easy GPU passthrough.)Lingering questions I&#x27;m working to figure out:- Will 2 RTX Pro 6000s run on a 1600 watt PSU? Not sure how much higher I can go without calling an electrician. (standard US home.)- Assuming I plop this into my home office, should I expect the PC to run significantly hotter than my current rig? (3960x threadripper, 128GB RAM, 1600watt psu, overclocked and watercooled 4090.) My water temp, measured at radiator, is about 60c at peak load. (This is the only number I care about, as this is what I have to consider to be comfortable sitting next to it.)

Post-hoc justification. There&#x27;s no analysis of whether that level of hardware was necessary to launch, only that they did get that hardware and did launch.

At the end of the article, the author has this to say:&gt; UPDATE: Launch was a success! 400K+ views, and multiple companies reached to use my IP. Read more here[0][0]<a href="https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distribution-fine-tuning&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distri...</a>

This article appears to lack any reason for &quot;needing&quot; this beast, or any real comparison with alternatives, both of which are required to answer the question posed in the title. It&#x27;s a summary of how much they spent and some light anecdotal comparison to what they might have spent on cloud services, but clearly they didn&#x27;t do an exhaustive hunt for value.The real question is whether or not they could have done whatever it is they did with less hardware. Is there a business idea here that could have been proven on cheaper hardware that could be upgraded as demand increased? Is the expected ROI there based on future earnings?Absent any indication that this was needed in the first place, I can only conclude that it wasn&#x27;t worth anything.

Sounds fun&#x2F;stressful&#x2F;rewarding. I&#x27;m most interested in the update at the end though &#x27;Launch was a success! 400K+ views, and multiple companies reached to use my IP.&#x27; I too, like probably 1 in 5 of the people reading this, think I have figured out some major problems with LLMs (context and computation research) but have wondered the best way to &#x27;release&#x27; and get value out of it. I can see training being a little easier in that you release weights against a known model arch but not the training code. Wy stuff is all custom layers though. Any thoughts on a release strategy where you need to release the layer code for people to see test weights&#x2F;the benefits?

UPDATE: Launch was a success! 400K+ views, and multiple companies reached to use my IP. Read more hereIt seems that he managed to get what he wanted from the hardware and I&#x27;m happy for them.He said something interesting at the beginning of his post, he compared the cost of the hardware to the cost of his time based on his FAANG salary. Which is an interesting way to think of this, but the rest of the article didn&#x27;t make me understand if at the end he did save money&#x2F;time based compared to just rend on the cloud.Also, outside of the power cost, hardware has other costs too, you need to operate it, maintain it, set it up, etc. all that require time. I mean, even the process of figuring out if it had a good enough ROI compared to cloud, takes from your time (collecting data, analyzing data, etc etc).

I have profited richly from my children, but not in a monetary sense.However much it has cost me monetarily, it has repaid itself ten times over in value to my very soul.

I (probably not obviously) meant the &quot;too much&quot; part, where kids fail to grow&#x2F;launch when (optional) things are given to them too easily.I didn&#x27;t mean food, shelter, medical, education, lego, others-where-required-by-law.

One of these things is not like the others. If you don&#x27;t spend the money on one of them, you can get a visit from government officials that might decide to take that &quot;item&quot; from you. You&#x27;d also be a worthless human to spend that money on the other 3 while not on the one.

Other things people spend &quot;too much money&quot; on:- muscle cars, with all the stuff, driven occasionally.- boats, that don&#x27;t get taken out much- gamer x, where x=system or laptop or keyboard or mouse or desk or glasses or mousepad or speakers or ... usually with &quot;&gt; too much RGB&quot;- children$48k for something constructive even if ai related? no problem, refreshing even.

That&#x27;s the case with Self-hosting anything. It is the privacy that matters.

Except this math is 10x too high (unless accelerated depreciation is all of it) - a million tokens at 28 tokens&#x2F;sec and 75W and 20c&#x2F;kwh should cost $0.15 not $1.50. (And less with MTP.)

It&#x27;s comparing laptops to dedicated GPUs in a server environment. The best comparison would be the Mac Studio but the current release is almost 2 years old at this point. We&#x27;ll see what a likely M5 Ultra Mac Studio looks like, probably in Q3 this year.But yes, for pure inference, the M5 Max Macbook Pros probably aren&#x27;t there yet. They have other utility though of course. And you can get 64GB and 128GB MBPs at a discount. Micro Center currently will let you buy a 64GB M5 Max MBP for under $4k currently, for example.

&gt; Why didn&#x27;t you take into account [...] the fact that a laptop can still hold a decent % of its resale value, and is useful for many other tasks than running an LLM?Because that wasn&#x27;t what they claimed to research?<pre><code> &gt;&gt; for inference it&#x27;s definitely not worth it.
</code></pre>
It&#x27;s entirely fine if you enjoy local LLMs on your computer, there are people doing horribly inefficient inference on smartphones now. But for pure inference tasks, it&#x27;s pretty obvious why M5s and Mac Studios aren&#x27;t replacing TPUs and GPUs.

Why didn&#x27;t you take into account batching, input tokens, different costs of electricity, and the fact that a laptop can still hold a decent % of its resale value, and is useful for many other tasks than running an LLM?

I did the math at least on a Macbook pro, and for inference it&#x27;s definitely not worth it.- <a href="https:&#x2F;&#x2F;www.williamangel.net&#x2F;blog&#x2F;2026&#x2F;05&#x2F;17&#x2F;offline-llm-energy-use.html" rel="nofollow">https:&#x2F;&#x2F;www.williamangel.net&#x2F;blog&#x2F;2026&#x2F;05&#x2F;17&#x2F;offline-llm-ene...</a>
- Discussion: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48168198">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=48168198</a>

&gt;&gt; To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade.The high cost and power consumption are both signs of the death of Moore&#x27;s law, so you are probably correct that this system will be near state of the art for some time.

Yes! It scared me too. I tried to insure it under my renter&#x27;s insurance policy, but they not surprisingly refused. I had to get business insurance to cover it

I have a second computer with an RTX 4090 for gaming (running Windows). I also used the new RTX 5090 running Linux to evaluate whether Proton&#x2F;Wine allow me to run Windows games on linux (yes, it works, but the compatibility and frame rate issues make me stick to native Windows for now).

Having built an almost identical rig earlier this year can promise at least one similarly-spec&#x27;d machine gets equal use between AI and gaming (Both on Linux). Stupid-excited for the Steam Frame to finally come out.

or crypto... what&#x27;s old is new again.

I would probably hate someone if they were buying the same hardware as me but doing something actually useful with it. Any game worth playing doesn&#x27;t require high specs anyway. There is such a large catalog of old games.

&gt; No wonder gamers hate AI bros.Personally, playing with AI models is way more fun than getting sucked into a game loop. Game loops feel like busy work hooked to an engineered dopamine drip. AI models are new frontiers and are exciting to build with, modify, lobotomize, and hack around with.

Not even a single mention of gaming.No wonder gamers hate AI bros.

I can&#x27;t imagine spending $48K on a home GPU server, but I did just splurge and buy a PC with an RTX 5090, specifically to hold the largest models you can fit in 32GB. It&#x27;s a top of the line PC with water cooled high end CPUs, 64GB RAM, RTX 5090 for $5K. To me the jury is still out whether this was a worthwhile investment, but I do expect to use this machine for a decade. I don&#x27;t run it at 100% power (it&#x27;s mostly idle, except for times when I&#x27;m training or doing batch inference). It has the nice property of being blackwell generation, similar to the machines we use at work.It just scares me to own a box that is $48K in my house, especially if it breaks, or gets stolen.

I think the relevant issue is you could conceivably have a house with two outlets with opposite phases. Bussing them together in the PSU will then create a short

I guess it was supposed to be a humorous aside, but it wasn&#x27;t actually helpful because the relevant issue is when you pull more total amps from a single circuit than it&#x27;s fused for (usually 15 or 20 amps in U.S. residences). The failure mode is usually tripping the circuit breaker.That issue can often be addressed fairly easily by splitting the power draw between two adjacent circuits. You can have an electrician do it permanently or temporarily DIY it with an appropriately rated extension cord. The real issue was OP was in an apartment at the time so an electrician would have been difficult. I assume they decided to just have a system integrator build it because they didn&#x27;t want to figure out how to segment and route the power rails in a dual power supply system, but it&#x27;s not exactly rocket science. Problems are often more due to choosing power supplies that aren&#x27;t up to their claimed spec, not pre-testing them under load or using incorrect or under-spec cables.

Probably means hiring someone who has more knowledge about PSUs and especially about having two simultaneous PSUs. There are questions like: when you press the power button how do the two PSUs turn up and in what sequence? How do you deal with the PWR_OK signal? What if there are voltage differences between the two PSUs? What about power backfeeding?

I read this as; the &quot;professional PC builder&quot; would carry some sort of insurance. So it isn&#x27;t really &quot;safer&quot;, but if something goes wrong, the investment is (potentially) still safe.Just an assumption, though!

&#x27;If you google “plugging a PC into multiple outlets”, you get lots of warnings that if you even consider such a setup you will instantly burst into flames. So I hired a professional PC builder make sure it was safe.&#x27;Not really sure how that makes it safe but OK!

You kind of bury the lede in that Article, it&#x27;s a good article, well done getting interest in your work.Will you now be selling these GPUs for a profit?

I did this with used parts and cheaper consumer cards (3090s) and did much of the same calculations. I found it was way cheaper for me as well.The main advantage, however, is that the friction of &quot;this is going to cost me in tokens to even try&quot; goes away. I was so much more willing to take chances and try new things on my own hardware than I would have been if I were paying API costs. I feel like this point isn&#x27;t made clearly enough by those of us who run these absurd self-hosted inference systems.Thanks for the write up, was a fun read. I spent an order of magnitude less, but I could relate to your story from beginning to end.Epyc (Milan), 512gb ram, 4x 3090

Hi! Thank you so much for posting this! I got back luck&#x2F;timing when I tried, so happy it made it to the front page! (I am the author)

Agreed. And the gained time either goes toward 1) more experiments, or 2) leisure, which makes you sharper in the lab and happier overall.
Not sure the &quot;I saved $17,000 so far&quot; framing is the most useful way to look at it, but it&#x27;s a cool project and I love that people are doing this kind of thing.

This is a difficult calculation to make because you wouldn&#x27;t rent time on the exact same system in the cloud. Depending on what you&#x27;re running, a bigger server with better inter-GPU interconnects in the cloud might complete the task so much faster that the additional per-hour expense is more than covered.

The $48K also isn&#x27;t fully sunk cost - there&#x27;s a non-trivial residual value for those GPUs at the moment and likely for a few years yet. The server has a depreciation curve that&#x27;s pretty enviable, actually!

Shallow take: They made an LLM that uses fewer emdashes.Cynical take: They made an LLM that can bypass existing AI slop detectors.Realistic take: They found a research problem they found interesting, dumped a bunch of capital and sweat equity into and (claimed to have, at least) found a solution. Neat!

They have a subsequent post (from Monday) about what they&#x27;ve been working on: <a href="https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distribution-fine-tuning&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distri...</a>(I would assume they haven&#x27;t made a lot of $ off of this, if nothing else because they&#x27;ve only just put out that post and demo. They do seem to have produced a model that doesn&#x27;t sound very LLM-y to my ear, though it also seems rather weak for its size.)

I am not the author, but he has been training&#x2F;tuning? a model that produces text that mimics the source material in a more natural way. So getting the LLMs to produce less bland and boring LLMisms, according to the following up blog post.

citing from the article:&quot;I spent a long time trying high risk&#x2F;high reward experiments and failing. But now I have something good. I’ve solved a major problem with LLMs. And I’m launching next Monday so we will soon see if it’s actually a breakthrough or just LLM psychosis &quot;Maybe ai companies today have some bounty program?

Just curious OP (if you&#x27;re the one posting) -- what do you mean by independent researcher? What are you researching and are you making $$ from it or are you living off previous built up savings? Seems like an interesting path. What research have you looked into so far?

Stuff like this + OpenClaw with Mac Minis a while back is sort of exposing a probable local AI flywheel waiting to happen.Someone needs to solve proper distribution of packaged GPUs with some Tesla-like wall connector for a consumer grade box that is plug and play.Maybe John Ternus ends up doing that at Apple since they sit closer to this consumer profile.

Quick tip for people who want to experiment with local models: A lot of the common smaller models are also available on openrouter or other services. Dirt cheap.I know it&#x27;s not the same. But a lot of people buy expensive GPUs, just to find out they have no real use for smaller models.

Giving an LLM samples and tell it to apply the style in the sample works a lot better than just telling it to copy a style it may have seen, or a list of rules.They do it well enough that it&#x27;d take really good output to beat.

If nothing else, rosmine&#x27;s DFT [1], which is what they were working on with this setup, seems like a worthwhile investigation.While I&#x27;m skeptical that there is much of a moat, at least for the large players, it should at least hopefully set rosmine up with for the next job :)It does seem to fix the current biggest issues with using LLMs for writing at various publishers. If you&#x27;re The Economist, you have a very specific house style and you have a decent corpus of articles written in that style. At least on my reading of it, rosmine can use DFT to get a model to closely match its outputs, in terms of the language quirks that are generated, to that of the corpus it is fine tuned on. ie it will very much match the house style, particularly as it is used in writing, vs giving a system prompt to an LLM that has some Economist articles in its vast training set, and telling it to write in that style- it will do an ok job, but still exhibit LLM language quirks despite itself. Even if you feed it the specific &quot;style guide&quot; that they give their authors, I dare say the reality of their writing is the best place to learn, and it sounds like DFT can ground the writing of a model in a specific corpus like that.[1]: <a href="https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distribution-fine-tuning&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rosmine.ai&#x2F;2026&#x2F;05&#x2F;18&#x2F;fixing-llm-writing-with-distri...</a>

From the author’s POV it seems like they were going to do this research regardless, so this is asking what the most cost-effective way to do that research.Or, for a person who did have a great way to monetize the same workload they’d probably find a lot of value in reading this post.

So the answer is: &quot;TBD if I can actually make money to pay this back&quot;

I think their retrospective at the end of the article is grounded and logical:&quot;If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center&quot;I&#x27;m sure there are use cases when renting makes sense, but it can get crazy expensive really fast if you&#x27;re not careful.

FYI: If you&#x27;re in a similar situation, think very carefully before you build your own. The $17000 might sound like a lot; but when you take into account your time and risk tolerance, renting might be a much better solution.

I agree :) Also, I heard Teknium trained the original Hermes model on 2x 4090. You can do a lot with a little compute

(For reference I’m talking about the DFT post from the same blog.) I love that ML is still in the “gentleman researcher” stage where relatively small amounts of startup capital can buy a ticket into frontier research.For a lot of research questions 6 GPUs is even overkill.It’s one of the reasons I’m skeptical of the “trillion dollar supercluster” idea [0]. I think what we need is more reasonably smart people investigating medium-sized problems. A “GPU middle class” you might say.[0] <a href="https:&#x2F;&#x2F;situational-awareness.ai&#x2F;racing-to-the-trillion-dollar-cluster&#x2F;" rel="nofollow">https:&#x2F;&#x2F;situational-awareness.ai&#x2F;racing-to-the-trillion-doll...</a>

He didn&#x27;t consider the possibility of renting it out during the downtime to Vast.ai to make some money back.

Jensen Huang said &#x27;the more you buy, the more you save,&#x27; and you actually took it personally.

I suspect a standard 20A 110V circuit can probably handle 2x RTX 6000 Pros. 15A probably can but that requires more research.During initial setup of the server I am putting together, I found that a machine with 4x Blackwell cards derated to 300W can get by on a single 120V 20A circuit. It&#x27;s tight but doable. A lot depends on the power supply. I don&#x27;t think it&#x27;s a great idea to run 4 high-power GPUs on a single ATX-style PSU, even a beefy 1600W job.The other questionable part is whether all four cards can temporarily spike at full power during boot, before the wattage limit is applied by the OS. Some accounts say this is possible, and if so it could shut down the party in a hurry. But I didn&#x27;t see any misbehavior when I tried it.

Cloud servers have cheaper electricity, the scale of industrial-level cooling, no issues for you (as a user) with hardware failure (ie you just use a different server; it&#x27;s not your problem) and can amortize their cost by running 24x7. I&#x27;ve seen H100 computer hours for as little as $2.As the author notes, there are also electrical&#x2F;wiring issues that cap how much compute gear you can run in a space not designed for it. I suspect a standard 20A 110V circuit can probably handle 2x RTX 6000 Pros. 15A probably can but that requires more research. Anything more than that and you&#x27;re using multiple circuits, which has issues, or you need an upgraded circuit (eg 40A 240V) with all that entails (eg heavier duty cables, custom plug, etc).

The idea is similar to maintaining on-prem vs cloudCloud is optimized for development velocity but its nature of high margin business eventually makes on-prem more promisingIt could be too late but it might be worth looking into tax saving if you have a business. Depreciation of asset is a loss and may deduct your income. (I&#x27;m NOT a tax expert)

And here I felt like I was wasting money on an Intel B70 to run LLMs locally.

Depends what you want to do and which cards you have, but usually going with any older (3rd gen+) threadripper pro setup will give you a lot of pcie lanes.I myself run with gigabyte trx40 aorus xtreme, but since it&#x27;s regular threadripper (not pro) with 4 GPUs 2 of them will run at x16 and two of them at x8 speeds

I know. But this is a forum filled with technical professionals and I would like to get actual opinions from actual humans.AI is cool but it&#x27;s not going to have all the good and bad experiences that humans have had with different motherboards.

You could ask AI and get pretty far reading the answer.

I have four old 24gb Nvidia cards. They&#x27;re not great but they&#x27;re not useless either. The problem is that I haven&#x27;t really figured out a good way to actually use them.Genuine question; would anyone here recommend any specific motherboard to best utilize these cards?

I am also considering to buy 3-4x RTX 6000 Pro 96GB plus some Ryzen workstation with a grant.Is this the best general-purpose choice as of 2026 with $50k for training, fine-tuning and running large open models?

I have the Macbook M5 MAX with 128 GB of RAM. I put its performance at roughly equivalent to the RTX 5070 Ti. The M3 Ultra 512 GB for me is about half the performance of the RTX 5070 Ti but obviously it has the ability to do more because of the increased memory.I don&#x27;t think anything compares to the nVidia chips at all.

You would install a 240v circuit (in the US) like for an electric clothes dryer.Edit: I now see the author was in an apartment and couldn&#x27;t do this, so I concede this is not responsive here.

So some things have changed since this rig was first built (2024). The most relevant is that $6800 RTX 6000 Ada 48GB has arguably been supplanted by the $9500 RTX 6000 Pro 96GB.The Ada has a memory bandwidth of 960GB&#x2F;s. The Pro has 1.8TB&#x2F;s and about 40-50% better performance so is at least equivalent in processing power, much better in memory bandwidth (important for inference) and can hold larger models on a single card.I&#x27;ve considered buying a rig with 1-2 6000 Pros for similar reasons but I want to see what happens with this year&#x27;s Mac Studios with a likely M5 Ultra. Macs have a shared memory architecture whereas NVidia segments the market based on max memory where the biggest consumer card (RTX 5090) has 32GB of VRAM but still excellent memory bandwidth (1.8TB&#x2F;s). A RTX 5090 rig will still trounce a Mac Studio seems to be the conventional wisdom. Despite being able to hold larger models and being able to chain Mac Studios on TB5, their lower memory bandwidth (~900GB&#x2F;s) and lower overall GFLOPS mean they still come out behind.That being said, the current Mac Studios are relatively long in the tooth, being released in 2024.I&#x27;m still not sure any of this is really wroth it because things are still changing so fast. I think there&#x27;s a decent chance of a number of large AI companies going bust in the next 2-3 years such that you&#x27;ll be able to buy enterprise AI hardware at cents on the dollar, a bit like how Google bought data centers in the post-dot-com crash.But anyway, nowadays I&#x27;d be looking at the RTX 6000 Pro as the sweet spot, having anywhere from 1-4 in a single server.The electricial issues the author mentions are interesting. I hadn&#x27;t really thought about the max amperage on a residential circuit. In a DC, these would typically operate on three phase power and much higher overall amperage. I wonder if there&#x27;s a device you can buy that can combine multiple residential circuits into a single power source for a server this power hungry?

&gt; paid a professional pc builderThey did not. That&#x27;s a mining rig not a workstation. It&#x27;s visible from the photo and the chart showing multiple failures over a short period of time including the risers -- which are visibly very low quality -- failing twice.You have 50K, you call a real expert like Puget Systems or Digital Storm.

I wonder why using 2 PSUs resulted in having slower interconnect.There is no specs in this blogpost regarding cpu&#x2F;motherboard choice, but if you go with threadripper pro they have 128 pci-e lanes for some time now, so using all GPUs at full speed shouldn&#x27;t be a problem

If you split models using pipeline&#x2F;layer parallelism you don&#x27;t have to care about a slow interconnect, you&#x27;re just slowed down a lot when running a single inference at a time as opposed to a fully pipelined minibatch. But tensor parallelism requires much faster interconnects than you could get in your average server, so I&#x27;m not sure that a different motherboard would help all that much.

A guy on Facebook with more confidence and better insurance

what is a &quot;professional pc builder&quot; in 2026

Did you think about Max-Q cards? 300W and they aren&#x27;t that noisy either, 14% lower perf than non-Max-Q card.

Consumer motherboards can still make sense even if you leave some performance on the table. Running an actual 8x GPU server is not something you&#x27;d want to do in an apartment. Imagine the old Lucasfilm &quot;THX&quot; trailer where an unearthly-sounding foghorn whine rises to a sweeping crescendo at reference level, only without the decay at the end.At the time he put this rig together, there weren&#x27;t a lot of open-weight LLMs that could run well on 6x48=288 GB, so it probably wasn&#x27;t a huge loss. There still aren&#x27;t, really.Right now I&#x27;m in the process of cramming Blackwell cards into an old DDR4-based Milan server, where the important thing is to be able to run large models at all. The GPU fans alone burn over 400 watts at full throttle.

Nope, they don&#x27;t support it. And afair even if they did, you would be limited to connecting only in pairs, not all 6 together

Don&#x27;t those Ada 6000 GPUs support NVLink? I think I can even see the cover for the connectors in OP&#x27;s pic.edit: Hm, finding mixed information online on whether that&#x27;s still supported or not. Apparently it was removed in workstation GPUs.

&gt; Because of this I got a motherboard with slow GPU interconnect. It’s good for running many small experiments in parallel (which is my main use case) but horrible for any models split across gpus.:( you paid a professional pc builder and you weren&#x27;t told this?

The research that&#x27;s presented in another article on the same site is way more interesting than the betteridges law article linked here. It&#x27;ll be very useful in my own latest project if this research is incorporated into some model I can rent by the token!

Yes, i mean, he could rent a cage and run grumbl it there. It doesn&#x27;t have to be a standard datacenter server, even though a standard datacenter server would be better and cheaper.

the very last line of the article:&quot;If I were to do this again, I wouldn’t do a custom build like this. I would buy a standard datacenter server and rent space in a colocation center. But then I would miss saying Hi to grumbl once in a while.&quot;

The cheaper, easier solution would probably be just to get an electrician to wire up a high amperage 240V outlet just like your electric stove or dryer has, and then get a PSU that connects to that.Would probably cost you $500-1000 depending on how difficult your home is.

out of curiosity, did you check how much would cost to rent a cage in a colocation space? Having to power your computer from two different outlets sounds wild..

&gt; Proper protection costs thousandsFrankly that&#x27;s something a landlord should provide. And there&#x27;s insurance against losses from electrical issues.

Also a lightning strike or surge from the electric utility could fry the whole rig. Proper protection costs thousands, and even then it&#x27;s not guaranteed to protect everything

It doesn&#x27;t cover risk. If one or more gpus dies, who pays for it? If you rent, you are guaranteed to be insulated from this risk. But owning, you might not have the best return policy from the vendor. And if you are actually at fault for breaking it, they have every right to deny a return. Or if your apartment is burglarized or catches fire (possibly from overloading the circuit) you are out the entire investment.

It is unsafe for wires to be handling higher power than it was rated cause the wires act like very low ohm resistors. At some high enough I, you’re still gonna be generating power P=I^2R which is mainly thermal and melt the wires.

&gt; I thought that I could not get a standard datacenter server because my apartment wouldn’t let me upgrade the circuits, so I needed to have 2 power supplies plugged into different circuits.Why didn&#x27;t they just put a higher amp breaker in the box?

Was my $48K GPU server worth it?

Project Hail Mary – Stellar Navigation Chart

Using Kagi Search with Low Vision

Blog ran on Ubuntu 16.04 for 10 years. I migrated it to FreeBSD

AI is killing the cheap smartphone

Google Is Shattering Under Its Own Weight (The IBM-Ification of Google?)

We're testing new ad formats in Search and expanding our Direct Offers pilot

For the past 5 years or so I&#x27;ve been working on a ground-up redesign of Freenet, my peer-to-peer project from the early 2000s (now renamed Hyphanet).The new Freenet has been up and running since December along with some early applications like River[1], our decentralized group chat and Delta - a decentralized CMS. Users have already started to build their own apps on Freenet including games, and we have some interesting apps in development like Atlas, a search&#x2F;recommendation engine.Architecturally, this new Freenet is a global, decentralized key-value store where keys are webassembly contracts which define what values (aka &quot;state&quot;) are valid for that key, how or when the values can be mutated, and how the state can be efficiently synchronized between peers.We&#x27;ve developed a unique (AFAIK) solution to the consistency problem, every contract must define a &quot;merge&quot; operation for the contract&#x27;s associated state. This operation must be commutative, meaning that you can merge multiple states in any order and you&#x27;ll get the same end result.This approach allows state updates to spread through the network like a virus[2], which typically achieves consistent global state in a few seconds or less.Like the world wide web, Freenet applications can be downloaded from the network itself and run in a web browser - similar to single-page apps on the normal web. However, rather than connecting back to an API running in a datacenter, the webapp connects locally to the Freenet peer and interacts with Freenet contracts and delegates over a local websocket connection.If you&#x27;d like to try Freenet we have convenient installers for the major desktop OSs but not yet mobile, and you can be chatting with other users on River within seconds[3]. Happy to answer any questions, you&#x27;re also welcome to read our FAQ[4], or watch a talk I gave back in March[5].[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;freenet&#x2F;river" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;freenet&#x2F;river</a>[2] <a href="https:&#x2F;&#x2F;freenet.org&#x2F;about&#x2F;news&#x2F;summary-delta-sync&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freenet.org&#x2F;about&#x2F;news&#x2F;summary-delta-sync&#x2F;</a>[3] <a href="https:&#x2F;&#x2F;freenet.org&#x2F;quickstart&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freenet.org&#x2F;quickstart&#x2F;</a>[4] <a href="https:&#x2F;&#x2F;freenet.org&#x2F;faq&#x2F;" rel="nofollow">https:&#x2F;&#x2F;freenet.org&#x2F;faq&#x2F;</a>[5] <a href="https:&#x2F;&#x2F;youtu.be&#x2F;3SxNBz1VTE0" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;3SxNBz1VTE0</a>

Show HN: Freenet, a peer-to-peer platform for decentralized apps

Uv is fantastic, but its package management UX is a mess

Indexing a year of video locally on a 2021 MacBook with Gemma4-31B (50GB swap)

Spotify will start reserving concert tickets for fans

Python 3.15: features that didn't make the headlines

Hey HN, We&#x27;re Gus and Carlos from Runtime (<a href="https:&#x2F;&#x2F;runtm.com">https:&#x2F;&#x2F;runtm.com</a>). We&#x27;re building infra that lets your whole team (including non-engineers) ship with Claude Code, Codex, and other agents without engineering having to handhold every session.After Mentum (YC S21) was acquired, I personally shipped 4 full-stack products in 3 months using coding agents. When I tried to roll the same workflow out to the rest of the team, it fell apart: Most PRs were unmergeable slop - Every repo required an engineer doing one-off local setup. - Skills and context lived in one person&#x27;s head. - There was no safe way for a PM to touch a real codebase without risking a bad deploy or a secrets leak.Carlos comes from building agentic reconciliation systems at Modern Treasury and had a similar experience when letting his support team use devin.We ended up building internal background agent infra but it quickly became a nightmare to mantain and develop. We built Runtime so you don&#x27;t have to do this kind of thing.Runtime work like as follows. Engineering defines the context once: system instructions, skills, and scoped integrations installable via CLI, mise, npm, or any package manager. Then Runtime snapshots your full running environment including multi-service Docker Compose setups, Kafka, Redis, seeded DBs, so it comes up in milliseconds with every server already running.We orchestrate across sandbox providers like E2B, Daytona, EC2 or self-hosted K8s depending on your setup. Secrets are injected through our managed proxy so they never touch the agent directly, and guardrails run at the infrastructure level: command allow&#x2F;deny lists, network egress controls, and RBAC scoped per human and per agent. Every session also gets a shareable preview URL, so internal builds go from sandbox to the rest of the team without needing production access.Runtime works with whichever agent your team already uses: Claude Code, Codex, Cursor, Copilot, Gemini, Devin. You can trigger sandboxes from our web app, CLI, Slack, Linear, GitHub, or API.One of our customers built an on-call inspector that wires PagerDuty, Sentry, and their repo so when an alert fires, the agent finds the cause and opens a PR with a unit test before anyone gets paged. Another runs a finance agent in a private Slack channel pulling from Stripe, NetSuite, and Snowflake to run reconciliations in minutes with source rows attached.A fintech unicorn and several YC scaleups are live on Runtime, including a few teams who had built similar infrastructure internally and handed it to us to take over.The core is open source at <a href="https:&#x2F;&#x2F;github.com&#x2F;runtm-ai&#x2F;runtm" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;runtm-ai&#x2F;runtm</a>. Hosted version is live at <a href="https:&#x2F;&#x2F;app.runtm.com">https:&#x2F;&#x2F;app.runtm.com</a>, free tier included. We&#x27;re charging a flat platform fee plus compute, no token markup.Check our demo: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=wLwj__aEEh4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=wLwj__aEEh4</a>We&#x27;d love to hear how you&#x27;re thinking about the infra for letting more people across your org use coding agents without creating chaos!