Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It’s never been a good idea to batch ingest a lot of little single files using any ETL process on AWS, whether it be DDB, Aurora MySQL/Postgres using “load data from S3…”, Redshift batch import from S3, or just using Athena (yeah I’ve done all of them).


These weren't "little" single files... just separated by tld iirc.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: