I'm as bearish as anyone on the current AI hype, but this particular ship has sailed. Research is revealing these humongous neural networks of weights for next token prediction to exhibit underlying structures that seem to map in some way to a form of knowledge about the world that is, however imperfectly, extracted from all the text they're trained on.
Arguing that this is meaningfully different from what happens in our own brains is not something I would personally be comfortable with.
> Research is revealing these humongous neural networks of weights for next token prediction to exhibit underlying structures that seem to map in some way to a form of knowledge about the world that is
[[citation needed]]
I am sorry but I need exceptionally strong proof of that statement. I think it is totally untrue.
Yeah, that’s my thinking now as well. It’s going to take an incredibly long time but truly understanding each problem is probably the only way to go.
Which is where this beats self study using books, I think. With a book, I can sort of wing it and think I understand something when I only do so very superficially whereas when you do the problems you truly learn what you understand and what you do not. And MathAcademy is only problems, so …
I stand corrected. Can’t edit my original post, though.
Khan hides it well, though - it’s listed smack in the middle of sixth grade math and high school math courses. I skimmed through the list (again) and found some college math but not all. Maybe this is a new offering?
Not sure I’d want to use it as my only resource, but as supplementary material it’s excellent. He really explains concepts well (some better than others though, though this is likely a ymmv issue).
FWIW my experiences with MathAcademy roughly overlap OP’s: it’s really hard work and adult life seriously interferes with making speedy progress (notice their own success stories are with teenagers who can devote hours upon hours on racing through the - very good - curriculum).
They say 1 point is equivalent to 1 minute of work and that you should earn at least 45 points a day. Well, for me 1 point is nowhere near 1 minute of work: I’m sloppy and sometimes downright stupid so it’s 1,5 minutes at best and often much, much more.
Banging your head against a wall every day for more than an hour (sometimes much more) just to get to what they consider to be the minimum of 45 points is no fun, and probably even counterproductive. I managed to keep it up for four months and made reasonable progress during that time (on getting back to where I was at the end of High School, 30 years ago) but it also burnt me out. I’ve now scaled it back to 30 minutes (not points!) a day. As a result my progress is now glacial.
Also, they’re very much of the “just do lots of problems and you’ll learn mathematic concepts and principles by osmosis” school of math instruction. For me I had to buy a textbook to get some extra explanation.
The good thing is that the problems seem well thought out and the spaced repetition system definitely works (for me, anyway).
I’m going to keep it up, because I have enough disposable income to afford it (though it is much too expensive for what it is) and I really want to bring my math skills up to a level where I can follow along the math in ML papers (and also because math, it turns out, is kind of elegant and interesting). I could go the self-study route, but then I’d have to spend time and effort guiding myself and figuring out what it is I needed to work on. If nothing else, MathAcademy is good at taking care of this for you so you can focus on the math itself.
Any pointers on useful textbooks in this space? I seem to have difficulties finding one that is at the right level (not too easy, not too hard) or that provides a way to gauge your level and start accordingly at a later chapter or whatever.
I decided start with Calculus I on MathAcademy because that was the last thing I did in High School. MathAcademy disagreed and told me to do PreCalculus and even bits of Algebra II first, but I knew better (MathAcademy was right and in hindsight I should’ve just started the Foundation courses to build up my pretty weak algebra skills again).
For Calculus I simply use the textbook that’s recommended at the link above. As far as I can tell, it’s good. I don’t do the problems, though - for that I use MathAcademy.
I took college algebra three times - not because I failed, but because I had a couple multi-year breaks in my college career and didn't do much math during them. It was definitely worth it - each time I took the class I picked up on things I had missed previously.
If I went back for my master's, I'd probably take it again.
I'm biased, but very fond of the open-access introductory textbooks used where I studied. The department was very much pure maths, but the intro classes were accessible to general liberal arts students. I think the texts are relatively unique in that they're very proof oriented, yet with a pedagogical style that doesn't assume the reader is a future graduate student.
For linear algebra, we used the relatively standard Friedberg Insel and Spence [2], but I hear good things about https://hefferon.net/linearalgebra/.
[1] Link is off university domain, since apparently it was at some point turned into a bit more hardcore textbook oriented towards those going onto graduate studies in mathematics. If curious: https://www.amazon.com/Calculus-Analysis-Euclidean-Undergrad...
Not a textbook, but https://betterexplained.com is an awesome resource for gaining intuition, its author's approach is very unlike others I've encountered.
Just checking out a few of these will change your concept of what it means to understand something in math and cause you to seek out better explanations beyond the textbook one. You can also refresh yourself on a topic in a way that's fun.
For proofs and introductory real analysis, I highly recommend Prof. Jay Cummings' books at the awesome price of about $20 on Amazon for freaking 400 page books. If anything, just buy it to support the guy.
Yeah, I only have a goal of 25 points a day, Monday through Saturday, and I still usually take more than half an hour (though usually less than an hour) per day.
I actually feel that 25 points may be a bad choice as it makes me spend too much energy picking lessons that will barely add up to 25 so I can be done with my daily. Probably causes me to review or whatever when maybe I don't need to?
Worth it in the personal sense, or career-wise? I'd love to do a maths degree, but being mid-40s with family, mortgage, career etc makes it hard to see myself doing it for real.
They’re not a private tutor, though. They don’t explain very much and there certainly isn’t a way to ask questions.
As I said elsewhere, to me they’re about twice as expensive as they should be.
That's not really the case.
Each separate step of each lesson is explained and practiced many times. Repeated failures across multiple students are noticed and explanations reworked. If it's not enough, you can report your issues. And there are MA communities to check with if you really get stuck for some random reason.
The explanations are very limited compared to actual maths lessons, though: in my experience they were very often something like "it turns out that the formula for this is...".
IMO it's scaffolded and explained a bit more than an average mathematics lesson, though teachers vary a lot.
There's a whole lot of "here's the formula" and not so much "here's the derivation" in most classrooms.
The math classes that I taught: I tried to do a lot more of the why, either rigorously or using proof by gesticulation. But there were still absolutely times that I just handed something over and was like "do this, for now."
I’m currently doing the Calculus I course and while there are explanations interspersed throughout the problems, these mostly seem to be the bare minimum you need to work the problems. When I compare it to the calculus textbook I keep alongside it (Stewart’s “Calculus Early Transcendentals”) it barely seems enough.
Private tutors are much more expensive and not uniformly effective. Math Academy is an extremely low-risk bet for parents of math students (you'll know before the first usage period whether it's working out). I like the business model here a lot --- I also just think it's like something concocted in a mad scientists lab to annoy HN people, who always have a really hard time intuiting market/pricing segmentation.
You can, but you will spend a lot of time figuring out what it is that you need to study and where your weak points are. MathAcademy does that for you so you can spend your precious studying time on, well, what you need to study.
I think it’s very expensive, and the correct price should be €$25/ month at most, imho, but its spaced repetition system definitely provides value over self study.
You can discover your weaknesses yourself by doing problem sets then checking solutions. You'll notice what kinds of questions you keep getting wrong, then you make a note to study that area again or you do more problems in that area. You don't need a computer algorithm for this.
Right, and then you’re expending mental energy on figuring out how to teach math (to yourself) instead of on the math itself. This is not wrong, and will likely even teach you a thing or two (and in fact it was how self-teaching math worked before this came along) but, to me at least, MathAcademy seems to be more efficient in getting you to do just the math and nothing else.
You don't NEED anything really. But it's helpful to have a computer algorithm for this. Processing that yourself is meta effort not everyone has the extra time for or will be diligent enough about.
Interestingly, this is what the EU is. After WW2 it was decided to set things up so that European countries no longer could wage war against each other. Hence starting out with coal and steel (no war without those).
In such a setup, lack of democratic oversight is not a bug but a feature. It prevents democratically elected but possibly not entirely stable governments from doing something stupid. The drawback, of course, is that it also prevents democratically elected and intensely well meaning governments from doing the right thing if the supranational entity does not wish to facilitate that right thing.
This setup works (after a fashion) for a group of small and fairly weak countries, but for a single large and powerful country like the US there simply isn’t a supranational entity powerful enough to reign it in.
I'm a little sad that the 303 sound had such a short-lived and niche life. In the 90's it seemed to me like it was the first instrument that could challenge the hegemony of the electric guitar. It was so versatile - the sound is bouncy, melodic, and had some real "growl", all at the same time.
Unfortunately it really is a niche thing that only appears to speak to certain people.
As someone who feels like the sound of the 303 touches me deep in my soul, it's constantly disappointing to be reminded that other people don't hear it the same way I do. You can even see it in comments on this post where expressing a love or appreciation for the actual sound of the silver box is dismissed as elitist or something because lol whatever, any old synth sounds just as good. Most people either can't hear or don't care about what makes it special, which perhaps explains why it never became respected as a mainstream instrument like the 808 did.
Fortunately the clones these days are very cheap and very good and music has become so easy to obtain that you can visit Bandcamp every week and still find new tracks featuring the 303 and its descendants. Every now and then you might hear a 303 in a mainstream tune and it's a treat, but if you just love the sound and don't mind listening to music that few others get, I don't think there's ever been a better time.
I remember back in the 90s there was somewhat of a backlash against the 303, which presumably was part of what Norman Cook was getting at with the name of the song mentioned on the top of this thread. Ironically - or perhaps deliberately - that track was peak unimaginative/tedious usage of the instrument, which is funny because he had also done some much more elegant takes in his Pizzaman project.
For me it never felt like the 303 ever got really overdone in mainstream electronic music. Certainly there was the riff from Pump Panel's Confusion remix showing up all over the place, and there were a few tracks that got a high rotation on MTV like Daft Punk's Da Funk and Josh Wink's Higher State, but I don't think it was ever really ubiquitous outside of acid music, which is already a niche genre. Like, we never got 808s & Heartbreak for the 303.
It was definitely controversial inside the synth community, though, where hardcore analog and modular synth nerds scoffed at it being so limited and toy-like, and everybody - young and old - resented it becoming so expensive and sought-after, which in turn raised the prices of other vintage synths that according to the rumor mill could do a decent approximation if you programmed them just right.
Rebirth busted that market by making the basic essence of the sound available to everyone, and there were plenty of bad acid tracks that came out during that period, but I think that's also when the opportunity was there for it to really break through as a serious instrument. Later VSTs like Phoscyon took inspiration from mods like Devilfish and more elaborate clones like the FR-777, building on the 303 base to create the kinds of sounds that in the old days might have required a lot hacking/patching up of different instruments to construct. But by that point it was clear that the mainstream didn't really care.
I'm at work right now so don't have access to my music library to share specific favorite tracks, but there is still so much great music featuring the 303 coming out - it never stopped. There is stuff for people of every taste. If the more unsubtle stuff doesn't work for you, you might want to check out Mighty Force label, which has been putting out a bunch of IDM/braindance and pleasant electro music recently that sometimes has delightful uses of the 303. Also in the back of my mind for more IDM-ish and electro stuff are Analogical Force, Virtual Urban Records, HC Records, Nocta Numerica... There's a bunch more in that vein, plus all the usual suspects doing big room techno, hard party acid, all-hardware synth jams etc etc, but you probably need to dig in any case. I tend to find even the best albums only have one or two tracks that are to my taste, but everyone is different so it's great that there is so much out there.
In the 1980's nobody knew how to use synths beyond the default patches. That's why I think that 80's music sounds so generic and kind of hollow.
The 90's was different, the people making synth music pushed the synths past what their default setup was capable of. Synths used in the mid/late 90's for psychedelic/acid trance sound nothing like 80's synths, but they are the same synths.
The "303" was intended as a bass instrument, but with 90's acid trance it's typically used as a lead, as well as a bass.
Arguing that this is meaningfully different from what happens in our own brains is not something I would personally be comfortable with.