System Design Methodology - Simplified

Designing a complete system to solve a problem , goes hand-in-hand with knowledge of full-stack web development. This article will be a definitive guide for everyone who is keen to understand what skills one needs to conceptualize and design a system from scratch, leading up to its implementation.

The skill-set revolves primarily around being able to work on both front and back-end portions of an application. Front-end generally means the part of the application the user will be able to view and interact with, and the back-end is the one that handles logic, user authentication, server configuration, database interactions etc. 

One needs to make themselves capable of working and understanding instead of concentrating on being an expert on both the sides. Remember, technologies will replace older ones as they always have. So diving deep into something that might be irrelevant in a few years is not a great way to go. Think, broader instead of deeper.

You need to be strategic towards building any solution and accept the fact that dynamics will change throughout the project so not to have a very restrictive thought process. Because that has never helped, has it?

Below is the excerpt listing out skill areas one needs to build :

1. Basics of Technical Design

One needs to know what they are building. There are many open source tools for creating the phases of designs while in concept phase. I personally use draw.io (for drawing the flow diagrams of solution) , visio and ninjamock (for making wireframes sample below)

Steps to approach designing:

Outline use cases, constraints, and assumptions : Gather requirements of the problem. Clarify and jot down use cases, constraints, assumptions: Who is the user, How will they use it, How many users, What does the system do, inputs and outputs of the system, how much data do we expect to handle.

Create a high level design : Outline a high level design with all important components. Sketch the main components and connections.

Design core components : Dive into details for each core component. For example, SQL or NoSQL, Database schema, API and object-oriented design,

Scale the design : Identify bottlenecks considering the constraints. For example, whether you need load balancers, horizontal scaling, caching, database sharding.

Once you have cleared the flow of how the solution will progress, try to capture the components into the below system diagram. More details on each component will be covered as follows. Part are : DNS, CDN, Load Balancer

2. HTML/CSS

When starting to building the solution from front, we use HTML (to add content to a website running on the client) and CSS (to style your content). Covering the below sections would ensure you have basic understanding of both : Bootstrap, Semantic HTML , CSS preprocessors, CSS Box Model, CSS Media Queries.

3. JavaScript

The JavaScript language is the most popular language in both Full-Stack, Front-end, and Back-end Development. It’s the only language that runs natively in the browser, as well as a server-side language (with Node.js).

Understand how to work with JSON and how to manipulate it. Learn how to properly structure your code and modularize portions of it. One popular framework ( like React or AngularJS ) but again don't limit yourself to a specific language. (With a good understanding of JavaScript, picking a new framework to sit on top of it won’t be very difficult). jQuery code still exists in most applications and a solid understanding of it will be helpful.

3. Back-End Languages

After HTML/CSS and JavaScript, next is to master a back-end language that will handle database operations, user authentication, and application You should understand what is going on and you learn the nuances of your chosen language. A few are listed as below:

Python: Some popular frameworks for developing in Python are Django and Flask.

Java: It is an all purpose language for today to create any application overall (Spring , Hibernate)

Ruby: Some popular frameworks for developing in Ruby are Rails and Sinatra.

Node.js: It is itself a JavaScript environment which means you don’t need to learn a new language. So its great in combination with the Javascript language.

PHP: Just like Java, it is still very in-demand and should be understood thoroughly.

4. Databases & Web Storage

Storing data for later accessibility is a must in any application. Hence, one should have a good grasp on the following topics related to databases and storage :

Learn about NoSQL databases, e.g. MongoDB. Benefits of relational data, e.g. SQL. Know how to connect a database with chosen back-end language (e.g. Node.js + MongoDB). Web storage to store sessions, cookies, and cached data in the browser. Understand the benefits of in-memory data stores like Redis or memcached.

5. HTTP & REST

HTTP is a stateless application protocol on the Internet — it’s what allows clients to communicate with servers (e.g. your JavaScript code can make an AJAX request to some back-end code you have running on a server which will happen via HTTP). Some important topics you should learn about are listed below:

REST plus why its important with respect to HTTP. Best practices for designing a RESTful API. What are SSL Certificates. POST/GET requests. Learning how to use Chrome DevTools can be extremely helpful.

APIS

Making a rich web application today is much easier using Application Programming Interfaces or APIs. There are thousands of awesome APIs out there which will allow you to add on top of data and functionality other developers have built. Some more examples are as : here, some json APIs.

6. Web Application Architecture

To start creating a web application, you’ll need to know how to structure your code, how to separate your files and host them, store large media files, structure the data in your database, where to perform certain computational tasks (client-side vs server-side).

System design will pertain to the CAP theorem, before jumping onto its actual clustering. Decide which two out of these properties you want in your system (consistency, availability or partition tolerance). Below pointers should help you get started :

• PaaS, e.g. AWS. (offers dozens of products and services to help with storage, video processing, load balancing, content management)
• Performance optimization for applications and modern browsers. Another way to look at performance vs scalability: If you have a performance problem, your system is slow for a single user. If you have a scalability problem, your system is fast for a single user but slow under heavy load.
• Designing Web Applications by Microsoft.
• MVC.
• Latency vs throughput : Latency is the time to perform some action or to produce some result. Throughput is the number of such actions or results per unit of time. Generally, you should aim for maximal throughput with acceptable latency.

7. Git

Git is a version control system that allows developers working with a team to keep track of all the changes being made to a codebase and the timestamps when they were changed. It helps you get the latest code, update parts of the code, make fixes, and change other people’s code without breaking things.

• A reference list of some common git commands.
• A tutorial on using Git and GitHub for beginners.

8. Basic Algorithms & Data Structures

In web-development, even though focus on computer science topics like tree traversal, sorting, algorithm analysis, matrix manipulation, etc aren't really considered a point for survival. You technically can design a solution without these concepts. But some basic algorithms and data structures will help you analyze trade-offs. Some points to keep in mind:

• Hash Tables
• Trees and graphs being beneficial as data structures.
• Basics of Big-O analysis
• When to use an object vs an array and its trade-offs.
• Why caching is important when working with a large amount of data.

With these details at hand, and a lot of rigorous effort on learning to conceptualize your idea, you will be able to create a solution entirely. Most important to keep in mind, is technology is ever so changing. Only way to go ahead is having the basics of these concepts clear while having an urge to always learn new technologies.