Manju

Greetings In a previous article , we set up our environment to play with Elasticsearch. Let's try basic queries in this article by building a movie search. Unlike other databases, Elasticsearch gives full searching power. Let's slowly dig into that power. Hence this is not a deep dive. I use the top 250 movies from  imdb-api in this exercise. Movie database Let's build a movie database using imdb-api and play with queries. Save movies as documents Movies to contain text, numbers, arrays, full text Structured search using movie ratings, year Full-text search (title, actors) Phrase search (plot, title) Return highlighted search snippets Note: You can either use Kibana UI or a direct call to Elasticsearch. Hence I skip the URL path. curl -XPUT -H "Content-Type:application/json" localhost:9200/movies/_doc/tt0068646 -d ' { "title": "The Godfather", "year": 1972, "rating": 9.2 }' Indexing mov

Greetings When learning new technology, installation is the first burden and it is irritating. However, in modern-day development, we developers usually don't have to bother too much with operations and there are dedicated teams assigned for that purpose. Let's see how we can get started with Elasticsearch. For someone who starts exploring Elasticsearch, there are two main options. Cloud free trial Docker container Cloud free trial This is the easiest way for someone new to Elasticsearch. You just register for the free trial without a credit card. The only drawback is it expires in 14 days. But that is enough for initial experience. If you know how to use emails with plus at the end, you can have endless accounts. https://www.elastic.co/cloud/elasticsearch-service/signup Docker container The other option is to spin up a Docker container. That is also an easy option. However, you need to know just enough Docker to use it. With Elasticsearch 8 security is enabled by d

Greetings! When I started writing lambda and serverless, I wanted to design a real-world project. The example that was in mind was a geolocation extractor using an image upload that my friend did. I thought to tweak it to an Excel file upload with another example that is based on real-world e-commerce requirements. What I want to focus on here is the architecture. Seasonal offers Offers are common in any business. Now the marketing team decides that having a separate section to indicate seasonal offers with additional descriptions would increase the orders. As they are experts in excel they say hey Devs, we don't want a UI for us. We will give you an excel to make your lives easier. Requirement Design a restful API to expose seasonal offers that are coming as an excel sheet. Technical breakdown We can store the excel in our code and directly read it. But that is not an expandable solution. Hence we store the file in an S3 bucket and separate out the extraction logic. Architecture

Greetings We recently migrated our WordPress website into our microservices stack to maintain it within the same stack. It was a fun experience for me hence I'm sharing it with you so that you can learn from it. I will provide a high-level thinking process and the reasoning behind decisions. Note: The majority of this is still valid for non-WordPress websites as well. Option 1 - Direct database access WordPress uses MySql as the database to store posts. If you have full access to the database you can directly access it and fetch using your own queries. However, this was not our case luckily haha. Unless we would not get this learning experience. Option 2 - Selenium We started with the selenium web driver that runs through the Chrome web browser. Selenium gave us a perfect solution that scraped all necessary information. With selenium, we can navigate through pages, click on links and navigate to sub-pages. When we navigate to the desired page we can utilize selenium-provided APIs t

Greetings! Elasticsearch is a powerful tool that most companies use for various use cases. It is the best tool we can use to give users a seamless search experience. Before jumping in, let's try to understand it at a high level. We know that our data is there in our databases, however, we are not able to retrieve them faster mostly due to the spreadsheet-like data structure. Elasticsearch is document-oriented, meaning it stores data as documents as it is. It not only saves documents but also makes them easy to searchable. What is Elasticsearch Elasticsearch is a distributed, free, and open search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on top of Apache Lucine but hides the complexity behind a simple, coherent, restful API. We can name a few use cases. Search in websites Logging and log analytics Application performance monitoring Geospatial data analysis and visualization Security ana

Greetings! In my previous article, I created a Cognito user pool using the AWS console. However, that is not a good practice in real projects as it involves a lot of manual work. That is why we need to leverage infrastructure as code. In this article, we create the same setup using CloudFormation. It turned out that there is no proper example in the documentation and the documentation is also a little confusing. Had to do a little trial and error and dig the internet as well. What we need We need to create a user pool and register a client. After that, we need to retrieve an access token for the logged-in user. Steps We mainly need to configure; UserPool UserPoolClient UserPoolDomain Here is the complete CloudFormation template. AWSTemplateFormatVersion: "2010-09-09" Description: Cognito Stack Resources: MyWebAppUserPool: Type: AWS::Cognito::UserPool Properties: UserPoolName: my-web-app-user-pool Policies: PasswordPolicy:

Greetings! Movies are everyone's fun hobby. Designing a movie system is even more fun. We can design mainly 2 systems; System to support a single theater (or group) System to support all theaters What I am interested in in this exercise is the high-level architecture of a system to support all theaters, which is a movie platform. These sorts of systems are not built overnight. It takes a lot of engineering effort, designs, re-designs, etc to achieve the desired output. User requirements Our main requirement is browsing movies and booking a ticket. Other than that, we can add a few more useful features. Browse available movies in nearby cinemas Book a ticket with online payments Find movie information Notifications Recommended movies Comments and ratings User Registration Cinema registration We also need our system too; Better response time Searches by movie names Secure payments High-level design At a very high level, our system will look like this. We are going to explore and expa

Greetings! AWS Cognito is a serverless identity and access management server. We don't have to manage any server or database to handle user data and authentication, and authorization flows. In this article, we focus on how to create a Cognito user pool and retrieve an access token that can be used to access back-end data (we will not create a backend service though). Cognito User Pools Amazon Cognito User Pools makes it easy to create and maintain a user directory and add sign-up (user onboarding) and sign-in to your mobile or web application for authentication, authorization, and resource access and control. Let's get started. What we do Create a Cognito user pool Register a user and retrieve the authorization token Retrieve access token Create a Cognito user pool Go to AWS Console > Amazon Cognito to navigate to the Cognito page. Click on the "Create user pool".

Greetings! Triggering a piece of code by a timer has multiple use cases. Hence I am going to talk about how can we do it using AWS EventBridge. Run daily/monthly reports Send reminders Scheduled data mining Send marketing emails The source code for this article can be found here . Let's learn how we can achieve this architecture using AWS EventBridge and Lambada. EventBridge AWS EventBridge is a serverless event bus service that you can use to connect your applications with data from a variety of sources including AWS services and external services. An event is an indicator of a change in the environment. When an event receives, it applies a rule to route the event to a target. Rules match events to targets based on either an event pattern or on a schedule which is the focus of this article. Let's build the architecture using the serverless framework. Initialize the project Let's initialize the project using serverless cli. mkdir eventbridge-lambda-t

Greetings! I wrote about creating a CRUD API using API Gateway and Lambda in a previous article. However, I did not talk about how to secure it. In this article, I am going to explain how we can use Lamda Authorizer to secure an API. The complete code is here . Lambda Authorizer Lambda authorizer is an API gateway feature to control access to your API. It uses the provided lambda function to do so. This is useful when we want to implement a custom authorization scheme that uses a bearer token (JWT) or a request parameter. Token authorizer - uses a token to authorize uses. It receives the caller's identity in a bearer token, such as a JSON Web Token (JWT) or an OAuth token Request authorizer - uses parameters, headers, and query strings to pass user identity In this article, we use a token-based authorizer. For simplicity, we expect our token to be "cat". Initialize the project Let's initialize the project using the serverless framework.

Greetings File uploading and invoking logic based on that can be a common business scenario. We can categorize a few such use cases as below. Image resizing such as thumbnail creation Read JSON file content and invoke logic Image processing The source code for this article can be found here . Let's learn how we can achieve this architecture using AWS S3 and Lambada. S3 Events We can enable events for a particular S3 bucket for bucket operations such as file added, and updated. When we enable this feature, we need to provide the event recipient such as a lambda function. When the bucket operations occur, the provided target will be invoked. Let's build the architecture using the serverless framework. Initialize the project Let's initialize the project using serverless cli. mkdir s3-lambda-trigger cd s3-lambda-trigger serverless create --template aws-nodejs npm init -y npm install @aws-sdk/client-s3 --save We have installed the S3 dependency