Skip to main content

Posts

Showing posts from April 28, 2023

Design a serverless web scrapper

Greetings In a previous article , I explained how we synced the WordPress website into our microservices architecture. We use AWS EKS for our deployments and managing microservices. But what if we use a serverless architecture? Here is my attempt at the serverless solution on my serverless journey. Note that this is about converting your own website into a serverless architecture. Always check ethicality in use. Note: This is about the architecture, not about actual coding. Requirement Convert existing website into a serverless microservice. Technologies By sticking into Nodejs and AWS I would go with the below stack. AWS Lambda AWS S3 AWS Eventbridge AWS RDS AWS API gateway AWS Cognito AWS CloudFront AWS Amplify AWS SES Nodejs JavaScript Axios Cheerio React I would consider AWS Step functions in the future as one lambda is doing too many things (save in database, store images in S3, send emails). However, my AWS Step functions knowledge is not as great as today. Architecture Let's