What are the best practices for designing web applications with high availability and fault tolerance?

Microservices play a crucial role in designing robust web applications, offering benefits like streamlined deployments, enhanced modularity, efficient management, technology diversity, improved performance, granular cost control and much more. While the advantages of Microservices are undeniable, the imperative for these services to collaborate seamlessly in delivering customer value is crucial. It's essential to recognize challenges such as dependency management, intricate integrations, service orchestration complexities, network congestion, incompatibilities etc Architects must carefully weigh the decision. Transitioning from a monolithic to Microservices architecture demands thoughtful consideration of both advantages and challenges.

Forget bulky apps – build yours with tiny, independent helpers called microservices. Imagine it: small teams, each crafting one simple task like checkout or login. Like runners in a relay, they work together but if one stumbles, others keep the show going! Failures stay tiny, scaling is a breeze (just add more runners!), and updates are quick – fix one service, not the whole app. It might seem like more work at first, but these microservices are your secret weapon for web apps that run forever, just like a marathon champion.

There's a temptation to breakup the monolith. However, not everything warrants being a microservice, and doing so could result in a distributed monolith, so it's worth really understanding the problem you're trying to solve first. Could modularisation solve your problem? A good rule of thumb is, can I break this out from the monolith into it's own service so that it not only can have it's own development lifecycle but can serve many clients too. In my experience, the trick is to look for those services that you can make generic, processes that use a lot of resources and are becoming a bottleneck. A few examples that spring to mind, are a transactional email delivery service, a pdf generation service, and an address data service.

Microservices are being thrown out everywhere nowdays but sometimes it is miss-understood and leads to painful mistakes. Like anything in design, balance of requirement is key and the key to success. For example, based on separation of concern; If you design your microservices properly, it is easier to scale up/down the k8s container and lead to a smoother service overall

While microservices excel at building resilient applications, they are also a way to scale teams effectively. By embracing a microservices architecture with careful planning and the right organizational structure, you can reap the benefits of both highly available applications and empowered teams.

Here is some few points you must implemented: 1. Software Design should be modular not monolethic. For this you need to implement the Microservice approach for better performance and security and resuability of application modules 2. Application Should implemented according to SOLID prinicples. This would be helpful for flexibility and easy maintaibability. 3. Database Should be normalized as possible and having good database design is very critical role for application speed performance.

Implementing a microservices architecture involves breaking down a monolithic application into smaller, independent services. This promotes scalability, flexibility, and easier maintenance. Each microservice focuses on a specific business capability, enabling faster development and deployment cycles. Ensure effective communication between services, utilize containerization for isolation, and leverage orchestration tools like Kubernetes. Embracing a microservices architecture fosters agility and allows for more efficient scaling of your applications.

I've discovered that using a microservices design is really beneficial for scaling and maintaining huge apps. Each service operates independently, making it easier to update and repair without disrupting the overall system. It's similar to having a team where each member is responsible for a certain task; specialization increases efficiency and agility, especially in fast changing circumstances. Furthermore, it works well with cloud-based deployment, increasing flexibility and resilience.

Highly available and fault tolerant web applications generally support 99.99% or more web service availability. To reach these numbers, running multiple instances of web services in multiple locations is required. Web services are generally accessed through a single URL and incoming traffic is routed through load balancers. Load balancing is implemented by using DNS resolution or application load balancers routing the incoming traffic to web service instances according to a set of rules. In addition to load balancing, cloud services provide auto scaling functionalities to detect the health and capacity of the web services and scale them up/down based on autoscaling rules. These cloud-native tools are used to implement the best practices.

Load balancers are a really simple way to introduce scale to your application, however moving from a single application server to many is not always trivial. In a previous role, we had a single-server application that we wanted to move into the cloud, to load balance and ultimately scale. From memory, there were a few challenges with the application architecture that prevented that from being achieved. Ephemeral servers is key. One initial issue as that sessions and cache were stored on disk, so we moved this to a Redis cluster. Another issue was that generated files were generated on the wrong server, so we used sticky sessions until we could move the files into cloud storage instead.

Integrating load balancing distributes traffic efficiently across multiple servers, preventing bottlenecks and improving performance. Auto-scaling dynamically adjusts resources based on demand, optimizing efficiency and ensuring responsiveness. Utilize cloud services like AWS Elastic Load Balancing (ELB) and Auto Scaling or Azure Load Balancer and Virtual Machine Scale Sets for seamless implementation of these crucial components in your architecture.

Auto Scaling is important not only in performance but also for cost saving. A good example is dream11, the traffic bursts to almost 500 TPS for 45 mins before the IPL match starts. Else its 0 TPS. A similar case is with any data analytics system, in peak day time it may generate events ranging in millions per minute and come down to thousands in non peak hours. Keeping your service behind LB and ensuring to set auto scaling rules on clusters, ensures performance and availability. Non peak hours, you have less instances, less cost, 100% availability. Peak hours, you trade off performance and availability over infra cost.

Implementing load balancing and auto-scaling has been critical in my experience for dealing with traffic surges and assuring high availability. Load balancing distributes incoming requests over numerous servers efficiently, preventing any single server from getting overburdened. Meanwhile, auto-scaling changes resources dynamically based on current demand, seamlessly scaling up during peak periods and scaling down when demand declines. This combination has been critical in sustaining application performance and cost-effectiveness.

Also focus on additional aspects such as emphasize redundancy, failover events, and performance metrics. Communicate incident response plans, monitoring alerts, testing outcomes, and successful rollback procedures to demonstrate system resilience. Consistent formatting, action-oriented content, and feedback mechanisms contribute to effective communication within the DevOps framework.

There’s no certificate, role, set of tools or prescriptive process. There’s no specification, it’s not a product, or job title. There’s no one true voice on what DevOps is or isn’t. It’s about attitude, ideas, customs and behaviours. Culture, paradigms and philosophy. It’s a way of thinking, a way of doing and a way of being. Practicing as well as preaching. It’s a conversation. It’s about taking the best experiences and sharing those with others. There are a number of "g-forces" that helped me on that journey to really get behind DevOps, workout out what would help us work towards 10 deployments per day was key, and treating servers like cattle was a catalyst.

Adopting a DevOps culture involves breaking down silos between development and operations teams, fostering collaboration, and emphasizing automation. Implement continuous integration and continuous delivery (CI/CD) pipelines to streamline development, testing, and deployment. Embrace automation for infrastructure provisioning and configuration management. Encourage a culture of continuous improvement and feedback loops. By integrating development and operations seamlessly, DevOps practices enhance agility, reduce time-to-market, and improve overall software quality.

Like the old saying in Google, if you like it put a CI in it. Any company of any size should adopt this culture with environment pipelines. The benefits of having a CI (even a simple one) greatly enhances your development (and developer) productivity

You need to have an efficient devops setup. Making sure the application components have proper life cycle policies set. 1 - availability, auto scaling threshold setup, dynamic down scaling and up scaling of cluster capacity. 2 - if the system is taking a hit, devops process should have pager duties and an alerting mechanism. Failures happen, AWS team has the best engineers but in the end, they are engineers . Your system will go down, the point is how well were you prepared and were you prepared enough. At Paytm, we have realised that cost saving on infra is easy with proper devops process. Money leaks on cloud systems are huge, and the need for Devops practice setup is inevitable.

Adopting a DevOps culture improved the efficiency and collaboration of our workforce. We shortened deployment processes, enhanced product quality, and responded to market requests more quickly by merging development and operations. Continuous integration and continuous deployment (CI/CD) became industry standards, improving automation and minimizing manual errors. This transformation generated a more proactive, learning-oriented atmosphere in which responsibility sharing and feedback resulted in innovative solutions and a more resilient infrastructure.

Design fault-tolerant web applications by distributing architecture across multiple servers, employing redundancy through replication and backups, and implementing automated scaling. Use monitoring tools for continuous performance tracking, set up alerts for abnormal behavior, and incorporate the circuit breaker pattern for fault detection and redirection.Establish failover strategies with backup components. Additionally, deploy security measures, leverage global content delivery for optimized performance, and document fault tolerance mechanisms while providing training for effective handling of failures.

Wird eine Anwendung langsam, muss man nicht gleich den gesamten Code optimieren. Man kann auch versuchen, die Daten auf einen anderen Server zu verschieben. Häufig verwendete Objekte können gecached werden. Wichtig ist hierbei, dass man nicht alles cached, weil das gerade bei dynamischen Inhalten zu Problemen führen kann.

I would like to point out the fact here that "high availability" is being framed as one size fits all solution. I think it is should be framed more carefully to give the audience more context on what it means and what it is used for. For example, highly available database like Cassandra, would not fit the use case of a highly consistent database like Yugabyte. One example is a banking account would fit the bill of a highly consistent with ACID compliance, rather than a highly available system.

Read on the CAP theorem. Every DB is good for the specific needs and requirements. For example, Cassandra has no master slave config on nodes, so no master elections, availability is very good, but as there is no master, consistency takes a hit. On the other hand, mongo db has master slave config, master elections take a few milliseconds, availability takes a hit. You can use a write through cache for better consistency. In today's world, distributed DB are a need and to ensure a sweet spot in CAP and performance enhancements to ensure over ambitious performance, caching is something that helps you ensure main memory very very fast operation. Persistence can any time be set up for cache to bootstrap the system post failure.

Defensive programming as a practice does not improve code readability nor maintainability. It does, however, introduce complexity, by adding extra checks and repetitive logical conditions in multiple layers, implemented by possibly unreachable code and scenarios. To the maintainer, this can convey uncertainty in data or functionality, and therefore reduce trust, add code, and slow down the development, while not achieving the initial goal. I find solid and tested interfaces, including corner and negative cases, a better way to achieve trust and high availability.

Designing for observability and operability is paramount in large-scale applications. These characteristics empower SREs, developers and etc. to meticulously monitor, measure, and troubleshoot the application's performance, health, and behavior. Implementing tools such as metrics, logs, traces, alerts, dashboards, health checks, and incident management not only enhances high availability but also fortifies fault tolerance. The ability to swiftly identify and address issues is critical in these applications, ensuring a seamless user experience and bolstering customer satisfaction. In the fast-paced world, designing for observability and operability becomes a strategic imperative for sustaining reliability and trust in the digital ecosystem.

Apply lessons from service outage incidents! When you have implemented all the best practices and yet you meet with outage incidents, that's where interesting problems for your application/service are identified. And you don't just patch or mitigate, you setup your system to never repeat the same root cause, with effective RCAs and implementing repair items. For ex. You might identify human errors requiring you to build automation. Or you identify that the certs are not timely renewed for authentication. Or external services that your application interact with have limitations.

Before even architecting for high availability and fault tolerance, it is essential to understand how much is “high availability” and how tolerant the application should be for failures. Answering these two questions will help avoid having to over engineer or pre-maturely optimize a system. There is no “one size fits all” in architecture. A banking application can have different availability and fault tolerance requirements as against a food delivery platform. So, personally I feel understanding what is too much engineering saves a lot of effort and money - much before you actually start designing.

Self healing solutions! You cannot build a system that prevents 100% of errors. But you can build system that heals automatically and quickly in the event of errors. Design for recovery and not staying stuck. For ex. Reducing passive backend operations in the event of load and throttling to allow for active user operations. Restart services opportunistically when watchdogs alert. Turn off faulty app features through configuration settings while keeping service live.

I would consider additional factors such as: a. For high availability:Understand user load pattern (including concurrent users, user geography, business hours) on the system. b. for fault tolerance: recovery strategies at application level(availability of hot or cold backup) and infrastructure level

You need to talk about security concerns, when using microservices you need to take into account the network topology, who is connect to who, secret management, authentication, and more

A monolithic application will always be more available - and more performant - than a web of microservices. Microservice's main benefits are reduction of complexity, separation of concerns, individual deployability and scaling, graceful degradation. At the cost of performance and availability. If you need all microservices to work together for the site to work, failure of any is a failure of all. Choose your architecture for your requirements, and don't overlook the monolith.

One thing that most teams forget to do is to control complexity. The more things you have in the web application's 'chain of events', the more opportunities there are for a hiccup to occur. The same thing goes for the amount libraries and frameworks you bring in to the project. Each framework or library has its quirks and bugs and they can add up to less predictable behavior rather quickly. Only bring in what you need, what your main language and framework cannot feasibly handle. Don't be afraid to write code. Code you write, is code that you can test and directly verify. Simpler projects have less things that can go wrong and that tends to make them considerably more fault tolerant and that often results in higher availability.