Yep. I saw the title and got excited.... this is a particular problem area where I think these things can be very effective. There are so many data entry class tasks which don't require huge knowledge or judgement... just clear parsing and putting that into a more machine digestible form.
I don't know... feels like this sort of area, while not nearly so sexy as video production or coding or (etc.)... but seems like reaching a better-than-human performance level should be easier for these kinds of workloads.
Nick Zentner, a geology lecturer at Central Washington University, takes a particular subject and does a relatively deep, discussion oriented, dive into it over the course of 26 sub-topics... his "A to Z" series. In these he does a couple streamed shows a week and includes links to relevant papers and resources. At the end of each session is a viewer Q&A for those watching live. Almost an online continuing education course....
Of central importance to the first half of the current Alaska series is recent paper by geologist Robert S. Hildebrand titled: "The enigmatic Tintina–Rocky Mountain Trench fault:a hidden solution to the BajaBC controversy?"
What's great about these series is that he'll get a number of the geologists writing these papers involved in one way or another. Either contributing interviews or talks specifically for the video series, or like in the case of this Hildebrand centric work in the current series, Hildebrand himself is watching the stream and participating in the live chat with the other viewers, answer questions and the like.
Hell yeah!!! Huuuuge Nick Zentner fan here, he's the entire reason I even knew about it. I'm a PNW resident and love attending his lectures in April. If you can make it, please do!
You might have missed the big H2 section in the article:
"Recommendation: Stick with sequences, integers, and big integers"
After that then, yes, UUIDv7 over UUIDv4.
This article is a little older. PostgreSQL didn't have native support so, yeah, you needed an extension. Today, PostgreSQL 18 is released with UUIDv7 support... so the extension isn't necessary, though the extension does make the claim:
"[!NOTE] As of Postgres 18, there is a built in uuidv7() function, however it does not include all of the functionality below."
What those features are and if this extension adds more cruft in PostgreSQL 18 than value, I can't tell. But I expect that the vast majority of users just won't need it any more.
Especially in larger systems, how does one solve the issue of reaching the max value of an integer in their database? Sure for unsigned bigint thats hard to achieve but regular ints? Apps quickly outgrow that.
OK... but that concern seems a bit artificial.. if bigints are appropriate: use them. If the table won't get to bigint sizes: don't. I've even used smallint for some tables I knew were going to be very limited in size. But I wouldn't worry about smallint's very limited number of values for those tables that required a larger size for more records: I'd just use int or bigint for those other tables as appropriate. The reality is that, unless I'm doing something very specific where being worried about the number of bytes will matter... I just use bigint. Yes, I'm probably being wasteful, but in the cases where those several extra bytes per record are going to really add up.... I probably need bigint anyway and in cases where bigint isn't going to matter the extra bytes are relatively small in aggregate. The consistency of simply using one type itself has value.
And for those using ints as keys... you'd be surprised how many databases in the wild won't come close to consuming that many IDs or are for workloads where that sort of volume isn't even aspirational.
Now, to be fair, I'm usually in the UUID camp and am using UUIDv7 in my current designs. I think the parent article makes good points, but I'm after a different set of trade-offs where UUIDs are worth their overhead. Your mileage and use-cases may vary.
Idk I use whatever scales best and that would be an close to infinite scaling key. The performance compromise is probably zeroed out once you have to adapt ur database to a different one supporting the current scale of the product. Thats for software that has to scale. Whole different story for stuff that doesnt have to grow obviously. I am in the UUID camp too but I dont care whether its v4 or v7.
It's not like there are dozens of options and you constantly have to switch. You just have to estimate if at maximum growth your table will have 32 thousand, 2 billion or 9 quintillion entries. And even if you go with 9 quintillion for all cases you still use half the space of a UUID
UUIDv4 are great for when you add sharding, and UUIDs in general prevent issues with mixing ids from different tables. But if you reach the kind of scale where you have 2 billion of anything UUIDs are probably not the best choice either
There are plenty of ways to deal with that. You can shard by some other identifier (though I then question your table design), you can assign ranges to each shard, etc.
Because then you run into an issue when you 'n' changes. Plus, where are you increasing it on? This will require a single fault-tolerant ticker (some do that btw).
Once you encode shard number into ID, you got:
- instantly* know which shard to query
- each shard has its own ticker
* programatically, maybe visually as well depending on implementation
I had IDs that encode: entity type (IIRC 4 bit?), timestamp, shard, sequence per shard. We even had a admin page wher you can paste ID and it will decode it.
id % n is fine for cache because you can just throw whole thing away and repopulate or when 'n' never changes, but it usually does.
Yes, but if you do need to, it's much simpler if you were using UUID since the beginning. I'm personally not convinced that any of the tradeoffs that comes with a more traditional key are worth the headache that could come in a scenario where you do need to shard. I started a company last year, and the DB has grown wildly beyond our expectations. I did not expect this, and it continues to grow (good problem to have). It happens!
> For those that don't know its also built upon OTP, the erlang vm
This isn't correct. It can compile to run on the BEAM: that is the Erlang VM. OTP isn't the Erlang VM; rather, "OTP is set of Erlang libraries and design principles providing middle-ware to develop [concurrent/distributed/fault tolerant] systems."
Importantly: "Gleam has its own version of OTP which is type safe, but has a smaller feature set. [vs. Elixir, another BEAM language with OTP support]"
The comment you are replying to is correct, and you are incorrect.
All OTP APIs are usable as normal within Gleam, the language is designed with it in mind, and there’s an additional set of Gleam specific additions to OTP (which you have linked there).
Gleam does not have access to only a subset of OTP, and it does not have its own distinct OTP inspired OTP. It uses the OTP framework.
What's the state of Gleam's JSON parsing / serialization capabilities right now?
I find it to be a lovely little language, but having to essentially write every type three times (once for the type definition, once for the serializer, once for the deserializer) isn't something I'm looking forward to.
A functional language that can run both on the backend (Beam) and frontend (JS) lets one do a lot of cool stuff, like optimistic updates, server reconciliation, easy rollback on failure etc, but that requires making actions (and likely also states) easily serializable and deserializable.
You can generate those conversions, most people do.
But also, you shouldn’t think of it as writing the same type twice! If you couple your external API and your internal data model you are greatly restricting your domain modelling cability. Even in languages where JSON serialisation works with reflection I would recommend having a distinct definition for the internal and external structure so you can have the optimal structure for each context, dodging the “lowest common decimator” problem.
I understand your point, and I agree with it in most contexts! However, for the specific use case where one assumes that the client and server are running the exact same code (and the client auto-refreshes if this isn't the case), and where serialization is only used for synchronizing between the two, decoupling the state from it's representation on the wire doesn't really make sense.
This is also what really annoyed me when I tried out Gleam.
I'm waiting for something similar to serde in Rust, where you simply tag your type and it'll generate type-safe serialization and deserialization for you.
Gleam has some feature to generate the code for you via the LSP, but it's just not good enough IMHO.
Rust has macros that make serde very convenient, which Gleam doesn't have.
Could you point to a solution that provides serde level of convenience?
Edit: The difference with generating code (like with Gleam) and having macros generate the code from a few tags is quite big. Small tweaks are immediately obvious in serde in Rust, but they drown in the noise in the complete serialization code like with the Gleam tools.
> Rust has macros that make serde very convenient, which Gleam doesn't have.
To be fair, Rust's proc macros are only locally optimal:
While they're great to use, they're only okay to program.
Your proc-macro needs to live in another crate, and writing proc macros is difficult.
Compare this to dependently typed languages og Zig's comptime: It should be easier to make derive(Serialize, Deserialize) as compile-time features inside the host language.
When Gleam doesn't have Rust's derivation, it leaves for a future where this is solved even better.
"Elixir has better support for the OTP actor framework. Gleam has its own version of OTP which is type safe, but has a smaller feature set."
At least on the surface, "but has a smaller feature set" suggests that there are features left of the table: which I think it would be fair to read as a subset of support.
If I look at this statement from the Gleam OTP Library `readme.md`:
"Not all Erlang/OTP functionality is included in this library. Some is not possible to represent in a type safe way, so it is not included. Other features are still in development, such as further process supervision strategies."
That quote leaves the impression that OTP is not fully supported and therefore only a subset is. It doesn't expound further to say unsupported OTP functionality is alternatively available by accessing the Erlang modules/functions directly or through other mechanisms.
In all of this I'll take your word for it over the website and readme files; these things are often not written directly by the principals and are often not kept as up-to-date as you'd probably like. Still even taking that at face value, I think it leaves some questions open. What is meant by supporting all of OTP? Where the documentation and library readme equivocates to full OTP support, are there trade-offs? Is "usable as normal" usable as normal for Erlang or as normal for Gleam? For example, are the parts left out of the library available via directly accessing the Erlang modules/functions, but only at the cost of abandoning the Gleam type safety guarantees for those of Erlang? How does this hold for Gleam's JavaScript compilation target?
As you know, Elixir also provides for much OTP functionality via direct access to the Erlang libraries. However, there I expect the distinction between Elixir support and the Erlang functionality to be substantially more seamless than with Gleam: Elixir integrates the Erlang concepts of typing (etc.) much more directly than does Gleam. If, however, we're really talking about full OTP support in Gleam while not losing the reasons you might choose Gleam over Elixir or Erlang, which I think is mostly going to be about the static typing... then yes, I'm very wrong. If not... I could see how strictly speaking I'm wrong, but perhaps not completely wrong in spirit.
Ah, that’s good feedback. I agree, that documentation is misleading. I’ll fix them ASAP.
> Elixir also provides for much OTP functionality via direct access to the Erlang libraries.
This is the norm in Gleam too! Gleam’s primary design constraint is interop with Erlang code, so using these libraries is straightforward and commonplace.
Thanks for the clarification. I've read about Gleam here and there, and played with it a bit, and thought there was no way to directly access OTP through the Erlang libraries.
This can be just my lack of familiarity with the ecosystem though.
Gleam looks lovely and IMO is the most readable language that runs on the BEAM VM. Good job!
This is a new issue for sure, but what trust is left to erode in the first place?
We already are regularly asked to accept mere emotionally motivated opinion as information. Of course its relayed as "fact-based reporting", wrapped in mostly tidy narratives which wear the makeup of a journalistic endeavor. But look at which facts are amplified, suppressed, or omitted; or the adjectives used around the facts or which facts are only ever spoken by the journalists and which facts are allowed to be conveyed directly by the participants. Even the choice of which stories are newsworthy and which aren't plays a role.
Modern journalism is already largely a call to action for adopting a particular worldview rather than an effort to ensure that you have the best information in formulating opinions. The press, the government, and activists already have taken the stand that you are a fool that is better manipulated into "correct thinking" by your betters than left to your own devices.
Given all that, I would contend that most reporting is not there to inform you in the first place: its there to manipulate your emotions on a subject while convincing you that somehow you are now "informed". I would say under those circumstances that any trust that still exists in the media is already misplaced. The betting question is just another avenue exacerbating an already unreliable journalistic landscape.
I think your experience matches well with mine. There are certain workloads and use cases where these tools really do well and legitimately save time; these tend to be more concise tasks and well defined with good context from which to draw from. The wrong tasking and the results can be pretty bad and a time sink.
I think the difficulty is exercising the judgement to know where that productive boundary sits. That's more difficult than it sounds because we're not use to adjudicating machine reasoning which can appear human-like
... So we tend to treat it like a human which is, of course, an error.
I find ChatGPT excellent for writing scripts in obscure scripting languages - AppleScript, Adobe Cloud products, IntelliJ plugin development, LibreOffice, and others.
All of these have a non-trivial learning curve and/or poor and patchy docs.
I could master all of these the hard way, but it would be a huge and not very productive time sink. It's much easier to tell a machine what I want and iterate with error reports if it doesn't solve my problem immediately.
So is this AGI? It's not self-training. But it is smart enough to search docs and examples and pull them together into code that solves a problem. It clearly "knows" far more than I do in this particular domain, and works much faster.
So I am very clearly getting real value from it. And there's a multiplier effect, because it's now possible to imagine automating processes that weren't possible before, and glue together custom franken-workflows that link supposedly incompatible systems and save huge amounts of time.
I think I like Tim Maudlin's approach to the question. Time isn't all that mysterious: it's simply fundamental meaning it cannot be explained has being the sum of some things more fundamental. The argument is that's where the difficulty lies for many is they want to express time as though it's a composite, and it just isn't. So the best you can do is explain time by referencing those other aspects of existence which incorporate time.
Here's a much longer take from Tim Maudlin, "Tim Maudlin: A Masterclass on the Philosophy of Time" (https://youtu.be/3riyyEmWwoY?si=9aI-bETWcNpdjMW9), Tim Maudlin is Professor of Philosophy at NYU and Founder and Director of the John Bell Institute for the Foundations of Physics. The podcast is Robinson Erhardt's.
I like Tim, his arguments come across as an honest attempt to answer real questions. I feel as if the current crop of physicists look down on things that appear too simplistic.
Their math give an answer with artefacts they can’t match to reality so they keep probing the outliers of these absolutely bonkers ideas, and rabbit holes keep getting deeper & weirder.
Time doesn’t exist. Do the people writing this garbage live in a vacuum? Is it AI slop?
> Time doesn’t exist. Do the people writing this garbage live in a vacuum? Is it AI slop?
Maybe don't dismiss an idea as garbage before trying to understand it.
One of the ideas here is that time is an emergent phenomenon, like how temperature and gas pressure seem real at a macroscopic level, but disappear once you look closer. They simply describe the average kinetic energy of all the molecules in the area.
More of the state is not allowed than is... at least by geography.
Also, there's a practical element. If I have to specify where they can't go, the default position is they can go anywhere... if I inadvertently leave an area out of my black-list where it really ought to exist: the default is "permission granted". With a white-list, the worst case is a forgotten or neglected area can't be operated in as a default and the AV provider will have an interest in correcting.
But also politics. It's a very different message to say we're going to white-list a given AV operator to exist in different areas vs. black-listing them from certain areas.
You think avoiding these things are why people are thinking about homeschooling?!
In San Francisco where I live the public school system made the decision to not offer algebra until later for egalitarian reasons. Basically since they couldn't bring up the students that faired poorly in math, they delayed the subject for everybody. Along the same lines, they took the one high school dedicated to the highest achieving students and turned it into a lottery system rather than something earned.
Yes, of course you're right, kids will be competing on all those subjects. But the idea that public institutions are somehow the safeguards of fundamental academic achievement is just out of touch.
Of course, San Francisco public school's embrace of socialist/egalitarian drive identity politics is just one example of public education failure. Elsewhere in the US in these times, other school districts are being turned effectively into seminaries because the other political side has other doctrinal objectives. In neither case is learning how to think or how the world really works is important.
they create the feeling of belonging to something, a group, a social circle. you cannot recreate that at home. you ll see what happens after a decade going along this path. The feelings of being an orderly part of a group and partaking in group stuff cannot be recreated at home
You know, I'm relatively social and I don't even go to school let alone a public school! How do I manage it? I engage with social groups in other ways.
Homeschooling doesn't necessitate going into a dark room somewhere and closing/locking the door. Sure, some may do that... but majority simply don't. Suggesting that there aren't paths to good socialization outside of public school is just rationalizing a position and ignoring the reality that many home schoolers don't operate in isolation. We're a social species and we tend to seek out others. This is even true for homeschoolers who can collaborate or engage in all manner of social opportunities that exist amongst a coalition of homeschoolers or they can take advantage of the social activities which are available in most communities of any size. Again, we're social creatures and we'll give into that impulse. To suggest that has to be nurtured by a public institution or is somehow unavailable outside of a public school just isn't credible on its face.
As others have said all up and down the thread, you take measures to build that social circle and interaction. And I can tell you from direct personal experience... a public school offers positive interaction opportunities to some, but wholly negative and even damaging to others... public schools absolutely offer a social experience. Whether that's good or bad in my experience is at best a case-by-case call.
I don't know... feels like this sort of area, while not nearly so sexy as video production or coding or (etc.)... but seems like reaching a better-than-human performance level should be easier for these kinds of workloads.
reply