Hacker Newsnew | past | comments | ask | show | jobs | submit | JoelJacobson's commentslogin

In the article, the linked equivalent query is https://github.com/gregrahn/join-order-benchmark/blob/master... which is written using legacy comma-separated joins and a huge WHERE clause:

    SELECT MIN(an.name) AS cool_actor_pseudonym,
           MIN(t.title) AS series_named_after_char
    FROM aka_name AS an,
         cast_info AS ci,
         company_name AS cn,
         keyword AS k,
         movie_companies AS mc,
         movie_keyword AS mk,
         name AS n,
         title AS t
    WHERE cn.country_code ='[us]'
      AND k.keyword ='character-name-in-title'
      AND an.person_id = n.id
      AND n.id = ci.person_id
      AND ci.movie_id = t.id
      AND t.id = mk.movie_id
      AND mk.keyword_id = k.id
      AND t.id = mc.movie_id
      AND mc.company_id = cn.id
      AND an.person_id = ci.person_id
      AND ci.movie_id = mc.movie_id
      AND ci.movie_id = mk.movie_id
      AND mc.movie_id = mk.movie_id;
Cleaned up written as ON joins eliminating redundant quals:

    SELECT MIN(an.name) AS cool_actor_pseudonym,
           MIN(t.title) AS series_named_after_char
    FROM cast_info AS ci
    JOIN name            AS n  ON n.id         = ci.person_id
    JOIN title           AS t  ON t.id         = ci.movie_id
    JOIN aka_name        AS an ON an.person_id = n.id
    JOIN movie_keyword   AS mk ON mk.movie_id  = t.id
    JOIN keyword         AS k  ON k.id         = mk.keyword_id
    JOIN movie_companies AS mc ON mc.movie_id  = t.id
    JOIN company_name    AS cn ON cn.id        = mc.company_id
    WHERE cn.country_code = '[us]'
      AND k.keyword = 'character-name-in-title';
The keyword and company branches only control existence though; their row multiplicities cannot affect MIN. We can therefore optimize this using EXISTS:

   SELECT MIN(an.name) AS cool_actor_pseudonym,
          MIN(t.title) AS series_named_after_char
   FROM cast_info AS ci
   JOIN title AS t ON t.id = ci.movie_id
   JOIN aka_name AS an ON an.person_id = ci.person_id
   WHERE EXISTS
   (
       SELECT 1
       FROM movie_keyword AS mk
       JOIN keyword AS k ON k.id = mk.keyword_id
       WHERE mk.movie_id = t.id
         AND k.keyword = 'character-name-in-title'
   )
   AND EXISTS
   (
       SELECT 1
       FROM movie_companies AS mc
       JOIN company_name AS cn ON cn.id = mc.company_id
       WHERE mc.movie_id = t.id
         AND cn.country_code = '[us]'
   );
Shameless plug: We're working on a new proposed SQL feature to add explicit syntax for key joins: https://keyjoin.org Here is how the query could then be rewritten further:

    SELECT MIN(an.name) AS cool_actor_pseudonym,
           MIN(t.title) AS series_named_after_char
    FROM cast_info AS ci
    JOIN title AS t FOR KEY (id) <- ci (movie_id)
    JOIN aka_name AS an ON an.person_id = ci.person_id
    WHERE EXISTS
    (
        SELECT 1
        FROM movie_keyword AS mk
        JOIN keyword AS k FOR KEY (id) <- mk (keyword_id)
        WHERE mk.movie_id = t.id
          AND k.keyword = 'character-name-in-title'
    )
    AND EXISTS
    (
        SELECT 1
        FROM movie_companies AS mc
        JOIN company_name AS cn FOR KEY (id) <- mc (company_id)
        WHERE mc.movie_id = t.id
          AND cn.country_code = '[us]'
    );
Note: for this to work, I had to add referential constraints (aka "foreign keys") to the join-order-benchmark, which only had PRIMARY KEYs declared.


I wonder what a nontrivial multi-table query with joins look like in Acadia?


I was amazed by the presentation all the way up until 33:47, where the buggy program compiles and only errors out when the invalid access is executed. So apparently Fil-C enforces memory safety dynamically at run-time, rather than detecting these bugs at compile-time.

A new language designed around memory safety, such as Rust, can reject large and important classes of memory-safety bugs at compile-time. Rust does not catch everything at compile-time, like bounds checks and RefCell, which are checked at run-time(, and the problems due to unsafe and C interop like explained in the presentation.)

Still, IMO the comparison becomes a bit apples vs pears when bragging about how much more memory-safe Fil-C is than Rust. It would have been helpful to explain this important difference about run-time vs compile-time.

Very cool and useful anyway.


I thought it was very obvious throughout the video what was happening...

Rust has compile time checks to enforce some safety. In comparison Fil-C is a lot more memory safe, it's more comparable to being a software implementation of CHERI.

Unfortunately it comes with downsides: runtime performance, runtime enforcement, granular safety (the safety is around allocations).

It also comes with huge upsides: you can run C/C++ with little to no code changes. Imagine compiling nginx and the associated system libraries with Fil-C, the performance hit is probably acceptable and now the web server is memory safe.

Rust probably provides enough memory safety (even if it is not complete safety) in most circumstances though.


I'm not a native English speaker, I know "memory safe" has a precise technical meaning, but the word "safe" still feels a bit strange to me given that a memory-safety bug can make the program crash at run-time.

Sure, Fil-C prevents the bug from possibly being exploited, which is a huge improvement. But crashing can be a DoS attack, and if running a mission-critical system, it might not be an acceptable outcome.

I just feel the already very good presentation could have been made much better if it had put more weight on explaining these trade-offs.


Even in memory safe Rust, out-of-bounds array access also results in crashes.

The whole point of memory safety is that bugs can not be exploited. A memory safe program does not mean a crash safe program.


The main advantage of Rust is that you can rewrite GPL software and replace the license with MIT.


The author's whole schtick is purposefully rage-baiting Rust devs, so really not that shocking that he downplays the compile-time trade-off


It did fix the original "Postgres LISTEN/NOTIFY does not scale" [1] post's problem though, which was mentioned in an update of that article:

    Update: Fixed in Postgres core
    This commit has eliminated the bottleneck in the postgres core.
    Credit to Joel Jacobson and the core postgres contributors for resolving this.
[1] https://www.recall.ai/blog/postgres-listen-notify-does-not-s...


Back in 2024, I was trying to optimize PostgreSQL's NUMERIC data type, which is base-10000, using Karatsuba. The problem of finding the optimal threshold of when to switch to Karatsuba turned out to be really hard, since it depends on the size of both factors combined. After some hundreds of hours, I gave up, and started thinking about if there could be a simpler solution. I came to think about another idea I'd had before but abandoned, about 64-bit modernizing the digit base from 10k to 100M, but that would be a challenge due to existing data on disk. Desperate of finding a solution, I wondered if it could be fast enough to do on-the-fly conversion back and forth between base-10k and base-100M, and then realized that, yes, of course, it will be fast already for quite small N (testing shows already between 3-6 base digits). The trick basically reduced the N in O(N^2) into half, i.e. O((N/2)^2), with some O(2*N) cost for the conversion back and forth.

I had a lot of fun hacking on this idea together with the maintainer of the NUMERIC data type, and after two months the patch finally was ready and got committed:

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit...


A bit tangential, but the folks behind the GNU Multiple Precision Library (GMPLib) have the problem of choosing algorithms more or less fleshed out. They've got some fairly approachable manual pages[1] for the various algorithms they use as operand sizes scale up, where Karatsuba is only the second of six options in terms of operational complexity.

[1] https://gmplib.org/manual/Multiplication-Algorithms


This really demonstrates the utility of pulling in a library written by experts in the field the library handles.


This is a problem I have a lot in modern programming ecosystems: How do I tell the difference between a library written by a team of experts who have spent decades optimizing everything to do with the task and a library written by one guy that's an unnecessary straightforward wrapper over the obvious implementation?


Or increasingly, a library written by an LLM referencing a pile of such “one guy” projects of varying levels of suck (from actually good to good-got-it-is-full-of-suck).


What I have been seeing recently is having a great LLM rewrite the library leads to fewer bugs and also optimized for your use case. More expensive for sure than pulling something like epub.js but when the library has infinite open issues it can be a lot better.


1. Read the code 2. Don't. Encapsulate and put yourself in a position to swap out the implementation at will.

(Maybe you'll swap out an unnecessary dependency for a simple helper function of your own).


But if I followed that advice here, I would say "Obviously I don't need a library to multiply numbers" and miss the decades of research and optimization.


You defer to the advice of experts you trust. Which somehow have become harder to come by in terms of signal to noise than 20-30 years ago.


Here is the full pgsql-hackers mailing list thread where you can follow our work from initial idea to commit: https://www.postgresql.org/message-id/flat/9d8a4a42-c354-41f...


Love this, thank you for providing the context!


It's complicated. :-)

There is a nice picture of the "best" choice for different ranges of sizes of numbers to be multiplied at http://gmplib.org/devel/log.i7.1024.png

More context and explanation can be found at: http://gmplib.org/devel/


I was curious what the claim

"10-100x uplift in terms of speed compared to Postgres on things like regexp_matches()"

was about, so I checked, and DuckDB's regexp_matches() is not the same as PostgreSQL's regexp_matches(). DuckDB's version "Returns true if string contains the regexp pattern, false otherwise." [1] while PostgreSQL's "returns a set of text arrays of matching substring(s)" [2].

I think the closest think in PostgreSQL to DuckDB's regexp_matches() is `string ~ pattern` or `regexp_like(string, pattern)`.

[1] https://duckdb.org/docs/lts/sql/functions/regular_expression... [2] https://www.postgresql.org/docs/current/functions-matching.h...


This article made me think of a strange claim by Elon Musk at 07:08 in this [1] interview:

"Cooling is actually much easier in space than it is on earth. You can just radiate to the vacuum."

I don't think that follows. The radiator is only the final heat sink. You still need to move heat from very dense chips into a deployable, space-rated radiator, and handle pumps, loops, leaks, redundancy, radiation damage, replacement, eclipses, Earth IR/albedo, and launch mass.

[1] https://youtu.be/D_1j5dVWNYI?si=R77VeVKlRXRhaBk5&t=428


Radiators in space are a solved problem. The ISS has 70 kW of cooling via multiple radiators, using a dual loop water/ammonia system. The water loop cools the station and high-priority electronics, then the ammonia loop cools the water loop and transfers heat to the radiators, which release the heat out to space. There are additional radiators just for the solar panels.

AI sat mini can use a simpler single ammonia loop, since the ISS uses a water loop on the station side to avoid toxicity issues in case of a coolant leak.

It's a far simpler engineering problem to solve compared to other challenges SpaceX is facing (Starship, Raptor, Starlink).


Sorry, should have emphasized that it was the "much easier" part I didn't agree with in that interview.


I think we can't rule out the explanation that all the ideas of space data centers could be connected to a desire by some of finding additional applications for rockets that can transport stuff to space.


I think we can't rule out the explanation that all the ideas of space data centers could have been connected solely to a desire to pump SpaceX's IPO.


It also provides a post hoc rationale for rolling Elon's loss making businesses into SpaceX. The bull case for ODCs looks a lot like the bull case for space based solar power that Musk once called "the stupidest thing ever"...

That said, SpaceX aren't the only entity proposing ODCs, they're just the only ones promising they're going to make country-sized profits out of them...


Here is a tl;dr as well: https://keyjoin.org/tldr.html


Shame on The Netherlands: ~89% of homes still use natural gas in some way for heating [1], and their government are now "scrapping the obligation to purchase a heat pump in 2026" [2].

[1] https://www.cbs.nl/en-gb/news/2025/50/ever-more-gas-free-hom... [2] https://www.abnamro.nl/en/personal/specially-for/preferred-b...


Yeah big surprise that the populist government didn’t achieve anything and rolled back green initiatives. Good thing that they fell, sad that it took so long.


Stupid symbolic politics to own the greens. Good thing is that heatpumps are the most rational choice for new homes, so I don’t think much damage was done.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: