Manju

Greetings I was once tasked with setting up an earlier version of a database system for a demo. It took me a whole day to complete that task due to versioning issues in the required libraries. It was nothing but just pain. I lost valuable time as a developer that I can use for something more useful than struggling with setting up a database system. This is why cloud-based solutions are much better. AWS RDS provides us with many benefits over using traditional on-premises systems. OS upgrades and patching Automatic backups Replication High availability Security Multiple database options Hardware and networking So what is RDS? AWS RDS Amazon Relational Database Service (Amazon RDS) is a collection of managed services that makes it simple to set up, operate, and scale databases in the cloud. We can choose from seven popular engines. MySQL, MariaDB, PostgreSQL, Oracle, SQL Server, Aurora MySQL, and Aurora PostgreSQL. We can even deploy on-premises with RDS

Greetings Emailing is a ubiquitous feature in modern applications that requires a reliable service to meet our email needs. Luckily, AWS Simple Email Service (SES) offers a cost-effective and secure solution. AWS Simple Email Service AWS SES is an email solution that relieves us of all operational responsibilities. It is an email platform that offers a convenient and affordable method for sending and receiving emails using your own email addresses and domains. Can I do "that"? When considering the use of such a service, we typically contemplate these functionalities. Personally, I experienced these aspects when incorporating AWS SES into our recent email feature. Sending simple emails Sending fancy HTML emails Send emails in bulk Send dynamic emails Store template somewhere Use list Use conditions Send attachments Send to multiple recipients Absolutely! SES indeed supports all of these features. Moreover, it can also be utilized for automatic

Greetings In various software development scenarios, we inevitably encounter external dependencies, whether they are our own services or entirely third-party entities. Especially when it comes to microservices this is very common. Systems can fail or crash if we don’t plan and handle these scenarios properly. Having personally encountered such situations, I became increasingly curious to delve deeper into this. Note that there are no “exact” answers when comes to these kinds of situations. The best is to let the business decide which option to use as it will save a lot of time, engineering effort and of course money by not over-engineering. What can go wrong? Let's first categorize the scenarios that can happen when using external systems. Service Unavailable Service is not able to handle high load Service had implemented rate limits Service is slow All these will impact our businesses hence we need to carefully design our systems. However, we cannot say there is a perfect solution

Greetings In a previous article , I explained how we synced the WordPress website into our microservices architecture. We use AWS EKS for our deployments and managing microservices. But what if we use a serverless architecture? Here is my attempt at the serverless solution on my serverless journey. Note that this is about converting your own website into a serverless architecture. Always check ethicality in use. Note: This is about the architecture, not about actual coding. Requirement Convert existing website into a serverless microservice. Technologies By sticking into Nodejs and AWS I would go with the below stack. AWS Lambda AWS S3 AWS Eventbridge AWS RDS AWS API gateway AWS Cognito AWS CloudFront AWS Amplify AWS SES Nodejs JavaScript Axios Cheerio React I would consider AWS Step functions in the future as one lambda is doing too many things (save in database, store images in S3, send emails). However, my AWS Step functions knowledge is not as great as today. Architecture Let's

Greetings Modern applications are highly data-driven, and distributed and performance is a top priority. Redis has become one of those critical components in most applications. In this article, we will briefly look into Redis and how we can use it in a NodeJS application. "Power real-time applications with sub-millisecond latency" Redis Fast, open source in-memory, a key-value data store for use as a database, cache, message broker, and queue (from the doc). It stands for Remote Dictionary Server. We usually think of Redis as a cache because it is the most used use case. But Redis is more than a cache. As an example, Redis is an ideal option to store user sessions in web applications. Here are a few use cases but for more details better to read the documentation. Caching Session store Messaging Queue Gaming leader boards Geospatial Caching is a must in Microservices architecture to improve performance. We utilize Redis for this purpose in our e-commerce ap

Greetings In a previous article , we set up our environment to play with Elasticsearch. Let's try basic queries in this article by building a movie search. Unlike other databases, Elasticsearch gives full searching power. Let's slowly dig into that power. Hence this is not a deep dive. I use the top 250 movies from  imdb-api in this exercise. Movie database Let's build a movie database using imdb-api and play with queries. Save movies as documents Movies to contain text, numbers, arrays, full text Structured search using movie ratings, year Full-text search (title, actors) Phrase search (plot, title) Return highlighted search snippets Note: You can either use Kibana UI or a direct call to Elasticsearch. Hence I skip the URL path. curl -XPUT -H "Content-Type:application/json" localhost:9200/movies/_doc/tt0068646 -d ' { "title": "The Godfather", "year": 1972, "rating": 9.2 }' Indexing mov

Greetings When learning new technology, installation is the first burden and it is irritating. However, in modern-day development, we developers usually don't have to bother too much with operations and there are dedicated teams assigned for that purpose. Let's see how we can get started with Elasticsearch. For someone who starts exploring Elasticsearch, there are two main options. Cloud free trial Docker container Cloud free trial This is the easiest way for someone new to Elasticsearch. You just register for the free trial without a credit card. The only drawback is it expires in 14 days. But that is enough for initial experience. If you know how to use emails with plus at the end, you can have endless accounts. https://www.elastic.co/cloud/elasticsearch-service/signup Docker container The other option is to spin up a Docker container. That is also an easy option. However, you need to know just enough Docker to use it. With Elasticsearch 8 security is enabled by d

Greetings! When I started writing lambda and serverless, I wanted to design a real-world project. The example that was in mind was a geolocation extractor using an image upload that my friend did. I thought to tweak it to an Excel file upload with another example that is based on real-world e-commerce requirements. What I want to focus on here is the architecture. Seasonal offers Offers are common in any business. Now the marketing team decides that having a separate section to indicate seasonal offers with additional descriptions would increase the orders. As they are experts in excel they say hey Devs, we don't want a UI for us. We will give you an excel to make your lives easier. Requirement Design a restful API to expose seasonal offers that are coming as an excel sheet. Technical breakdown We can store the excel in our code and directly read it. But that is not an expandable solution. Hence we store the file in an S3 bucket and separate out the extraction logic. Architecture

Greetings We recently migrated our WordPress website into our microservices stack to maintain it within the same stack. It was a fun experience for me hence I'm sharing it with you so that you can learn from it. I will provide a high-level thinking process and the reasoning behind decisions. Note: The majority of this is still valid for non-WordPress websites as well. Option 1 - Direct database access WordPress uses MySql as the database to store posts. If you have full access to the database you can directly access it and fetch using your own queries. However, this was not our case luckily haha. Unless we would not get this learning experience. Option 2 - Selenium We started with the selenium web driver that runs through the Chrome web browser. Selenium gave us a perfect solution that scraped all necessary information. With selenium, we can navigate through pages, click on links and navigate to sub-pages. When we navigate to the desired page we can utilize selenium-provided APIs t

Greetings! Elasticsearch is a powerful tool that most companies use for various use cases. It is the best tool we can use to give users a seamless search experience. Before jumping in, let's try to understand it at a high level. We know that our data is there in our databases, however, we are not able to retrieve them faster mostly due to the spreadsheet-like data structure. Elasticsearch is document-oriented, meaning it stores data as documents as it is. It not only saves documents but also makes them easy to searchable. What is Elasticsearch Elasticsearch is a distributed, free, and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on top of Apache Lucine but hides the complexity behind a simple, coherent, restful API. We can name a few use cases. Search in websites Logging and log analytics Application performance monitoring Geospatial data analysis and visualization Security ana

Greetings! In my previous article, I created a Cognito user pool using the AWS console. However, that is not a good practice in real projects as it involves a lot of manual work. That is why we need to leverage infrastructure as code. In this article, we create the same setup using CloudFormation. It turned out that there is no proper example in the documentation and the documentation is also a little confusing. Had to do a little trial and error and dig the internet as well. What we need We need to create a user pool and register a client. After that, we need to retrieve an access token for the logged-in user. Steps We mainly need to configure; UserPool UserPoolClient UserPoolDomain Here is the complete CloudFormation template. AWSTemplateFormatVersion: "2010-09-09" Description: Cognito Stack Resources: MyWebAppUserPool: Type: AWS::Cognito::UserPool Properties: UserPoolName: my-web-app-user-pool Policies: PasswordPolicy:

Greetings! Movies are everyone's fun hobby. Designing a movie system is even more fun. We can design mainly 2 systems; System to support a single theater (or group) System to support all theaters What I am interested in in this exercise is the high-level architecture of a system to support all theaters, which is a movie platform. These sorts of systems are not built overnight. It takes a lot of engineering effort, designs, re-designs, etc to achieve the desired output. User requirements Our main requirement is browsing movies and booking a ticket. Other than that, we can add a few more useful features. Browse available movies in nearby cinemas Book a ticket with online payments Find movie information Notifications Recommended movies Comments and ratings User Registration Cinema registration We also need our system too; Better response time Searches by movie names Secure payments High-level design At a very high level, our system will look like this. We are going to explore and expa

Greetings! AWS Cognito is a serverless identity and access management server. We don't have to manage any server or database to handle user data and authentication, and authorization flows. In this article, we focus on how to create a Cognito user pool and retrieve an access token that can be used to access back-end data (we will not create a backend service though). Cognito User Pools Amazon Cognito User Pools makes it easy to create and maintain a user directory and add sign-up (user onboarding) and sign-in to your mobile or web application for authentication, authorization, and resource access and control. Let's get started. What we do Create a Cognito user pool Register a user and retrieve the authorization token Retrieve access token Create a Cognito user pool Go to AWS Console > Amazon Cognito to navigate to the Cognito page. Click on the "Create user pool".