When you have a giant monolith with the "load the world" endpoint, it can be tricky to pinpoint the the "load the world" endpoint (or, as is often the case, endpoint*s*) is what's causing issues. Instead, everyone just tends to think of it as "the x app having problems."

When you bust the monolith into the x/y/z, and x and z got the "load the world" endpoints, that starts the fires of "x is constantly killing things and it's only doing this one thing. How do we do that better?"

That allows you to better prioritize fixing those scaling problems.

There is a point when it all starts to make sense. But that point is when you go into billions worth business, hundreds of devs etc. And going there has large cost, especially for small/medium systems. And that cost is not one off - it's a day-to-day cost of introducing changes. It's orders of magnitude chaper and faster (head cound wise) to do changes in ie. single versioned monorepo where everything is deployed at once, as single working, tested, migrated version than doing progressive releases for each piece keeping it all backward compatible at micro level. Again - it does make sense at scale (hundreds of devs kind of scale), but saying your 5 devs team moves faster because they can work on 120 micoservices independently is complete nonsense.

In other words micoservices make sense when you don't really have other options, you have to do it, it's not good start-with default at all; and frankly Sam Newman says it in "Building Microservices" and so do people who know what they're talking about. For some reason juniors want to start there and look at anything non-microservice as legacy.

It sort of is.

It's not a perfect world. One issue with monoliths is that, organizations like to take a "if it ain't broke, don't fix it" attitude towards things. Unfortunately, that leads to spotty service and sometimes expensive deployments. Those aren't always seen as being "broken" but just temporary problems that if you pull enough all nighters you can get through regularly.

It takes a skilled dev to be able to really sell a business on improving monoliths with rework/rewrites of old apis. Even if it saves money, time, all nighters. It's simply hard for a manager to see those improvements as being worth it over the status quo. Especially if the fact of running the monolith on big iron masks the resource usage/outages caused by those APIs.

That often is, because new solitions tend to come with their own problems.

I can see an advantage regarding resource hogging, but the flip side is the extra point of failure of network calls in microservices.

Not saying which is better, but deployment is orthogonal to logical dependence and correctness.

For instance, in a shopping site, why should a crash in the recommendations engine result in a non-functional webpage (rather than a working purchase page with no recommendations)?

Personally I think microservices start to make sense when you have several hundred developers (an environment I'm currently keen to never enter again - $work has 5 devs and might one day have 6).

I agree that microservices are suitable for large organization, where the organization practically has multiple products (which could be purchased from a vendor or sold to another company).

Networks could have unexpected delays, routing errors and other glitches. At least with a monolith you can often find a stacktrace for debugging. I have seen startups that have limited traceability and logging when using micro services.

When a small startup has to manage "scalable" K8s infrastructure in the cloud, distributed tracing and monitoring is often not prioritized when you are a team of 5 developers trying to find a product market fit.

I am not against microservices (I work with them daily) but you just trade one type of stability problem with another

I love working with microservices at the scale of $WORK, but we're Big Tech. I can't imagine why a 5 person startup would want k8s and microservices. You don't need that scale until you have more than 2 teams, and you're pushing at the very least 15 engineers at that point and usually the sales and marketing staff to make that investment worth it.

You're going to have to explain what you mean by that a bit more... You surely cant mean it as it is written.

If the recommendations team write code that causes the OOM-killer to end their process, making them run it on separate infrastructure insulates your "main store team" from the bugs they write.

I imagine his logic was something like: "How can OOMs happen less often if you run more processes (possibly on the same machine)?", while your comment actually wants to say: "if a specific service is affected by an OOM, with microservices only that specific microservice goes down, since it's probably running on its own hardware".

Split up services where it makes sense and don't over do it. That's how I design my projects.

So if a database service is not available you simply return stale, cached data until the service is back up.

It depends on what you are scaling. I think microservices are fundamentally more scalable for deployment, since changes can be rolled out only to the services that changed, rather than everywhere. Unless your language and runtime support hot-loading individual modules at runtime.

The solution to this is to use the right tool, a build system that supports monorepos like Bazel. Bazel solves this problem wonderfully. It only builds / tests / containerizes / deploys (rules_k8s, rules_docker) what needs to be rebuilt, retested, recontainerized, and redeployed. Builds are much faster, developers have God like visibility to all of an organizations' code, and can easily grep the entire code base and be assured their changes do not break other modules if Bazel test //... passes. It is language agnostic so you can implement your services in whatever language best suits it. It allows you to more easily manage transitive dependencies, manage versions globally across your org's codebase.

Of course Bazel has a steep learning curve so it will be years before it is adopted as widely as Maven, Gradle etc. But in the banks I've worked at it would've saved them tens of millions of dollars.

Also git would need to catch up to large codebases. I think Meta released a new source control tool recently that is similar to git but could handle large monorepos.

> I disagree, in my opinion micro-services hinder scalability of deployment

…and of anything related to testing:

- Want to fire up the application in your pipeline to run E2E tests? Congratulations, you must now spin up the entire landscape of microservices in k8s. First, however, you need to figure out which versions of all those microservices you want to test against in the first place, since every service is living in a separate repository and thus getting versioned separately.

- Want to provide test data to your application before running your tests? Well, you're looking at 100 stateful services – good luck with getting the state right everywhere.

We keep object definitions in a separate repo to not duplicate them and generate a maven dependence from that. Pretty simple actually. We mainly run on Java thought. Would be more complicated when there are multiple languages.

Protobuf or Json schema could come to the rescue if needed.

Admittedly, it isn't a scalability problem you will run into right away.

But when you need to roll out an emergency fix, there is a big difference between deploying to thousands of servers that all have everything, and ten servers running a single service.

The silly thing is that something like the JVM (which we use) really wants to host monoliths. It's really the most efficient way to use resources if you can swing it. The problem is when you give the JVM 512GB of ram, it hides the fact that you have a module needlessly loading up 64gb of ram for a few seconds. Something that only comes to a head when that module is ran concurrently.

200 threads, 12TB of RAM, with a pipe upwards of 200GB/s. This isn't even as big as you can go, this is a reasonable off the shelf item. If your service doesn't need more than this, maybe don't break it up. :)

I believe that this level of service can no longer accurately be described as "micro".

Let alone - AWS Lets you get that machine for less than a junior engineer salary ($7.5 per hour, roughly equivalent to $32 of hourly wage = $67k)

Ruby, Python, PHP and Node.js startups have to figure out how to use the cores while C++, Rust, Erlang and Go have no issues running a single process that maxes out all 64 CPU cores.

So it does not work for FAANG companies but what do I care. I have the rest of the world to play with ;)

Instead, if you build an architecture that can scale the number of corporate users by adding cheap $2k pizza box 1u servers as the company grows, that's much more attractive. Also, you can keep your systems design flexible enough to recompile and take advantage of advancements in hardware tech every 18 months – this gives you better operating margins as your own business starts to grow.

As long as hype chasers in middle management don't get in the way after convincing themselves they too must be like FAANG with a few orders of magnitude less of a consumer base.

They want to be both the architect and the manager and anything you say would be over ruled and since they are the boss its hard to ignore them.

This service is a monolith because it has 10K code and it needs to be broken up.The product is at MVP and its rock solid on Java Spring and it hardly crashes. We are never going to lose data based on the design choices we made. None of that matters.

We need zero down time upgrade, when we had zero customers.

I don't think you actually understand what microservices are. You don't put a load balancer to load balance between different services. A load balancer balances trafic between servers of the same service or monolith.

Microservices mean the servers of different services run different code. A load balancer only works together with servers running the same code.

Uh - what?

>A load balancer balances traffic between servers

Correct.

> of the same service or mononlith

Incorrect.

Load balancers used to work solely at Layer 4, in which case you’d be correct that any 80/443 traffic would be farmed across servers that would necessarily need to run the same code base.

But modern load balancers (NGINX et al) especially in a service mesh / Kubernetes context, balance load at endpoint level. A high traffic endpoint might have dozens of pods running on dozens of K8s hosts having traffic routes to them by their ingress controller. A low traffic endpoint’s pod might only be running on few or one (potentially different) hosts.

The load balancer internally makes these decisions.

I hope this clears up your misunderstanding.

The split is inessential. You can just as easily have homogeneous backends & one big load balancing pool. Instances within that pool can even have affinity for or ownership of particular records! The ability to load balance across nodes is not, as you claimed, a particular advantage of microservices.

Are you replying to the right comment? I made no such claim.

The point is, if a client makes a request to a server, the response should always be the same, no matter where the load balancer sends the request to. Which means it should run the same code.

Nginx doesn't even mention route based or endpoint based load balancing in their docs. Maybe they don't consider it load balancing either.

https://www.nginx.com/resources/glossary/load-balancing/

Describes exactly what I’m talking about.

Just because Nginx is doing it, doesn't mean it's load balancing. It's an extra function tacked onto a load balancer.

Network latency is real, and some systems really do run on a tight latency budget, but most sane architectures will just do a couple of calls to a database (same performance as monolith) and maybe a transitive service call or something.

Not at all. You can run your monolith on multiple nodes just as you can do with microservices. If anything, it scales much better as you reduce network interactions.

Little big concept or mono-micro people like the call it is where it is at. Spending too much time making a complex environment benefits noone. Spending too much time making a complex application benefits noone.

Breaking down monoliths into purposed tasks and then creating pod groups small containers is where it is at. Managing a fleet on kubernetes is easier than managing one in some configuration management stack. Be it puppet or salt. You have too many dependencies. Only your core infrastructure kubernetes should be a product of the configuration management.

Those machines are made very redundant.

All components in a Parallel Sysplex configuration can operate in redundant mode.

Redundant power. Redundant central processor complex. Redundant multichip module (what would be a chipset on microcomputer platforms). Redundant RAM Memory. Redundant Storage. Redundant internal interconnects. Redundant network adapters.

That's why corporations that require high availability to not get sued for millions/billions use them.

Geographically Dispersed Parallel Sysplex.

I'm not sure about that: you could still put something together within that budget, say, a few different VPSes across Hetzner and Contabo (or different regions within the same provider's offerings), with some circuit breaking and load balancing between those. Probably either a managed database offering, or a cluster of DBs running on similar VPSes.

Of course, this might mean that you have 1-3 instances of a service instead of 10-30, but as long as availability is the goal and not necessarily throughput, that can go pretty far.

Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.

  sed "s/amateur/comparatively poor, from a third world country, without VC money, or have cheap labor/g"

500 USD a month would get you approximately the following resources (taxes vary) on the aforementioned platforms:

  Contabo
  Nodes: 15 to 83 (depending on configuration)
  CPU: 150 to 332 cores
  RAM: 664 to 900 GB
  SSD: 4150 to 6000 GB
  
  Hetzner
  Nodes: 7 to 110 (depending on configuration)
  CPU: 86 to 192 cores
  RAM: 192 to 384 GB
  SSD: 2200 to 4800 GB
  
  (this includes regular VPS packages, not storage optimized ones, or dedicated hosting etc.)

> Otherwise what you have is a budget that is lower than the possible implications of temporary downtime. That doesn't make sense in the real world.

This (depending on the circumstances) does sound like a good point! Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.

But it leads me back to my original statement - extreme requirements for uptime don't come out of nothing.

If you're in a location where IT related labor is extremely cheap - you're just going to have people keep one server up.

I know I used to do exactly that, because the server was more than my annual income. But that didn't last long. After the first 20 minute downtime, the budget for HA solution was allocated. But before a certain point downtime wasn't expensive.

Non-profits would probably be the only reasonable exception, where HA and low budgets could coincide. Otherwise - nah...

> Maybe "the real world" isn't the best wording, though, and choosing "enterprise settings" or anything along those lines would be more suitable.

This is the point - for-profit corporations, by definition, don't want to waste money, and if "high reliability" isn't required (or they don't know about it), they don't waste money. Most of the time, they don't waste the money.

However, if "being cheap" would means billions in lost income (and they know about it), they really want to have reliable, redundant infrastructure and systems around.

I've almost never seen situations where a single request would need more resources available, than the entire server has (outside of large GPT models for text, though maybe that's because I couldn't afford beefy machines for that myself).

Instead, if your monolith needs X resources to run (overhead) and Y resources to serve your current load, then in case of increased load you can just setup another instance of your monolith in parallel with another set of X + Y resources (same configuration) and it will generally almost double your capacity.

Now, there can be some issues with this, such as needing to ensure either stateless APIs or sticky sessions, but both are generally regarded as solved problems (with a little bit of work). Monoliths themselves shouldn't be limited to running just a single instance and aren't that different from a scalability perspective than microservices.

Where microservices excel, however, is that you can observe individual services (e.g. systemd services or them running in containers) better and see when a particular service is misbehaving or scale them separately, as well as decrease that X overhead since each service has a smaller codebase when you have lots of instances running. This does come at the expense of increased operational complexity and possibly noisy network chatter, especially if you've drawn your service boundaries wrong.

However, at the same time I've seen actual monoliths that can never have more than one instance running due to problematic architecture, so therefore I propose the following wording (that I've heard elsewhere):

  SINGLETONS - a monolithic application that can ever only have a single instance running, for example, when business processes are stored in memory for a bit, or have user sessions or something like that stored locally as well; these will ONLY ever scale VERTICALLY, unless you re-architect them
  MONOLITHS - applications that contain all of your project's logic in a single codebase, although multiple instances can be launched in parallel, depending on your needs; can be scaled BOTH VERTICALLY and HORIZONTALLY; they have more overhead though and observability can be a bit challenging
  MICROSERVICES - applications that contain a part of the total project's logic, typically across multiple separate codebases, possibly with shared library code, pieces of your project can be scaled separately, BOTH VERTICALLY and HORIZONTALLY; they are operationally more complex, can involve more network chatter and while you can observe how services perform, now you need to deal with distributed tracing

I don't actually expect anyone to use these particular terms, but I dislike when someone claims that monoliths have the issues of these "singleton" applications when in fact that's just because they've primarily worked with bad architectures. Sometimes they wouldn't need to shoot themselves in the foot with microservices if they could just extract their session data into Redis and their task queues into RabbitMQ. Other times, microservices actually make sense.