I use Llama.cpp and OpenAI-style API of course (who doesn't, unless you use Ollama perhaps?)
Ollama sucks, they dropped ROCm support for my card out of the blue, and the Vulkan replacement wasn't ready. The developer just shrugged and didn't care about the people affected so I moved to llama-server. Also, here's some other reasons to avoid ollama: https://sleepingrobots.com/dreams/stop-using-ollama/
And really, ollama are trying to sell their cloud platform and are always running behind llama.cpp in new features like the KV Cache quantisation.
So numbers, I don't really measure it from day to day, I'm not really interested in benchmarks at all. But I just did a quick test, llama 3.1 8b q8_0 does about 70tk/s (with Vulkan up from 45 when I was still running ROCm), and qwen 3.5 9b q8_0 does about 45, I think that includes its time spent thinking, not sure. Overall I'm happy with that.
Smaller sizes and quants are of course faster than this.
It's not terribly much faster than a Mac Studio, but it's one hell of a lot cheaper of course. And for the price of a Mac Studio you can get a lot faster hardware. The only thing where the studio excels is if you need large amounts of vram like 64GB and up and run the models to go with that (I run multiple small ones and spread them over multiple cards)
Why are mathematicians a kind of programmers? Besides applied maths, aren't they more researchers that explore and discover, in contrast to the majority of programmers who are more like handymen?
Metamathematics, by Kleene, is programming in maths. Theoretical computer science is maths. A lot of foundations work is programming. Coding itself is like an extended problem set from a maths class. LaTeX itself is programming.
The difference to me is one of directionality - maths research is seeing a far off island and getting there by hook or by crook; bridge, draining the swamp, inventing an airplane or boat, whatever it takes. Software engineering is like covering a plain with tiles - every feature is ultimately filled in and the underlying beauty is obscured by a fractal of complexity required by the ever growing requirements.
> Metamathematics, by Kleene, is programming in maths.
Ookay.
Back in the day i was confused by 'Linear programming', which is optimisation and has nothing to do with coding.
> every feature is ultimately filled in and the underlying beauty is obscured by a fractal of complexity required by the ever growing requirements.
Right. I would say Mathematics tries to unobscure (patterns in) nature. Engineering is creating tools, sometimes leveraging natural patterns.
But yeah, Fourier or Laplace definitely created tools, too.
The * in regular expressions is from that book by Kleene; it used to be called the Kleene star. And the book is all building up the apparatus of functional programs, recursion, implementations of arithmetic and logic, etc. it is a great book for programmers. But it is for basic computability theory, the Gödel incompleteness theorem, results on regular languages and so on.
Would you mind checking again? I haven't found any new comments that note the spliced video and old ones I have replied to as well got removed.
If someone knows a way to unremove comments, this is the link to one:
https://www.youtube.com/watch?v=eiDhbZ8-BZI&lc=UgzCu8FPM4CtR...
The comments on this site (HN) may be what most people are talking about. Youtube comments are too easily censored, so they probably will never manage to stick around.
I have years of experience, but I never had the time (or will) to take on some _very minor nuisances_ or different areas of dev far from my day job expertise.
LLMs solved this. I produced about 12 different things that "I needed" to improve aspects of my life.
Each single took between a few hours to 3 days, and most of them I use daily, they are the most used applications (mobile, desktop and web) for my family.
It is a game changer.
Personalized custom software would never really reach critical mass, LLM enabled it, this is the age of personalized software, egosoftware, llmware.
> Each single took between a few hours to 3 days, and most of them I use daily, they are the most used applications (mobile, desktop and web) for my family.
How have you done distribution and auth? I'm interested in doing something similar but not sure of the best way to approach that part, especially with the family/multiplatform angle as well.
"deployed on TailScale" is going to be quite the hit in this age!
I do think it is a bit scary to imagine software devolving into an unconnected / less connected state. Lots of apps but less protocols, everyone kind of frontiersing their own stuff has enticement and I love the can do itiveness. But I am scared for a major regression, of disconnection.
I do think atproto is an enormously positive hope here. I used to be super thrilled by platform as a service stuff, works like Apache UserGrid that gave a wide range of basic compute platform stuff, accounts, data stores, etc. Letting devs focus on the core competencies, on their value add rather than building boring account systems and what not felt like a huge win back then & expected to see soar.
I think atproto is a very interesting resurgence of something similar-ish, where users have their own Personal Data Stores that your app can read and write records from. And which serves as an auth system for your app! It makes social app building incredibly simple, and there's been so many neat simple apps people have booked up with so much less ceremony than what doing connected stuff used to require. Atproto creating a platform as a service (PaaS) is super cool & may really enable awesome new ways of building.
