Friday, January 12, 2018

Introduction to Node


Node.js is an open-source, cross-platform, JavaScript runtime built on Chrome's V8 Javascript engine that allows developers to create all kinds of server-side tools and applications in JavaScript. Node.js was developed by Ryan Dahl in 2009.
  • Javascript everywhere
  • Great performance
  • Asynchronous and event driven
  • Single threaded
  • Node package manager (npm) provides access to thousands of reusable packages.
  • Portable


Express is an un-opinionated, minimalist web framework for Node.js

Setting up the Node environment

  • Install Node
  • $ curl -sL | sudo -E bash -
    $ sudo apt-get install -y nodejs
    $ node -v
  • Install express generator
  • The express application generator tool generates an Express application skeleton.
    $ npm install express-generator -g
    (-g flag installs the tool globally)
  • Create skeleton project
  • $ express --ejs --git hellonode
    $ cd hellonode
    $ npm install
    $  DEBUG=hellonode:* npm start
    then go to browser and enter http://localhost:3000/
  • Enable server restart on file changes
  • $ npm install --save-dev nodemon
    then update the scripts section as below,
    "scripts": {
        "start": "node ./bin/www",
        "devstart": "nodemon ./bin/www"

    $ npm run devstart
  • Directory structure
  • Let's have a look at the generated files.
    >> package.json
    defines the application dependencies and other information

    express: express framework

    body-parser: parses the body portion of an incoming HTTP request and makes it easier to extract different parts of the contained information.

    cookie-parser: uses to parse the cookie  header.

    morgan: HTTP request logger

    ejs: view template engine

    nodemon: enable server restart

    >> www file
    application entry point

    >> app.js
    sets up the application with various settings and middlewares.

    >> routes

    >> views
    view templates

Congratulations!!! you just created your first website which runs using Node!!!

Layouts in express ejs


This is the 5th part of our Node EMS.

In my last post I used the same html layout for all the views. But it is still repeating. So one solution for it to use a layout engine. For express ejs we can use express-ejs-layouts.

$ git clone
$ cd node-ems
$ npm install
$ npm run devstart

If you like to check the changes done in this tutorial
$ git checkout layout-with-ejs


$ npm install express-ejs-layouts --save


in app.js we add following,
var expressLayouts = require('express-ejs-layouts');
app.set('layout', 'layout/main');

Then we create main layout file and use <%-body%> to generate final html. All other html files will have only the body content.
out final main.ejs

Creating views with EJS


This is the 4th part of Node EMS.

Express uses Jade/ Pug as default view template engine. But it is not pure html which is difficult for a non-ui guy.
We can change the view engine to EJS using --ejs option when creating the project.

Complete code for this tutorial,
$ git clone
$ cd node-ems
$ git checkout create-views


EJS is a simple templating engine that generate HTML with plain JavaScript. There is no special thing about that since it uses plain HTML and uses JavaScript like syntax to generate dynamic contents.
Supports includes, filtters, etc


<% 'Scriptlet' tag, for control-flow, no output
<%= Outputs the value into the template (HTML escaped)
<%- Outputs the unescaped value into the template
<%# Comment tag, no execution, no output
<%% Outputs a literal '<%'
%> Plain ending tag
-%> Trim-mode ('newline slurp') tag, trims following newline

Create views

We are going to use a bootstrap starter template to quickly create our views.
Go to bootstrap site and get a starter template

$ mkdir server/views/employee
$ mkdir server/views/partials
$ touch server/views/partials/footer.ejs
$ touch server/views/partials/header.ejs
$ touch server/views/employee/index.ejs
$ touch server/views/employee/create.ejs
$ touch server/views/employee/edit.ejs
$ touch server/views/employee/view.ejs

<!-- views/partials/header.ejs -->

<!-- views/partials/footer.ejs -->

<!-- views/index.ejs -->

Now start the server and navigate to http://localhost:3000/ see the new styles in action.

Now we can fill our employee views.
*** Please check the git repo for those html codes. There are very little EJS codes.

With that we have completed basic CRUD in our system. We can further improve this by adding validation, security and of course more features.

Creating a controller in Node


In this 3rd part of the tutorial, we are going to connect our employee schema with the controller.

(this is incomplete without views on next tutorial)

We are going to use mongoose's built in methods to access database and do our operations.

Update employee controller as follow.

Creating models with mongoose


This is the 2nd part of our Node employee management system. In this part, we are going to create our model.


MongoDB is an open source NoSQL database that stores data as documents in JSON like structure. A "collection" of "documents", in a MongoDB database, is analogous to a "table" of "rows" in a relational database.

This combination is extremely popular because it helps to use same structure in entire application. (thus creating stacks like MEAN)


Mongoose is an Object Document Mapper(ODM) for Node. (like Hibernate for Java). It allows us to access MongoDB commands simply and easily.

Initialize database

* You need to have installed Mongo in your computer.
go to the terminal and do this,
$ mongo
> use nodeems;
> db.employees.insert({
   firstName: 'Manjula',
   lastName: 'Jayawardana',
   email: '',
   password: 'manju'
> db.employees.find().pretty();

Install mongoose

This will add mongoose support to our project
$ npm install mongoose --save

Configure database

$ mkdir server/configs
$ touch server/configs/database.js

* update app.js to require database configuration

Create mongoose schema

$ touch server/models/employee.js

Then mongoose will provide multiple methods to access data.
save(), find(), findOne(), findById(), findByIdAndRemove() and man more.

See you in next tutorial.

Node employee management system


Let's do some practical thing using Node.

In this Node tutorial series we are going to build an Employee Management System.
get the complete code
$ git clone
$ cd node-ems
$ npm run devstart
navigate to http://localhost:3000

home page

employees page

Generate the project

$ express --ejs --git node-ems
$ cd node-ems
$ npm install
$ npm install nodemon --save
   update scripts: "devstart": "node ./bin/www"
$ npm run devstart
quickly check the setup: http://localhost:3000/

Let's modify the folder structure a bit.

$ mkdir server
$ mv routes/ server/
$ mv views/ server/
$ mkdir server/controllers
$ mkdir server/models

> you need to modify the app.js to pick routes, views from server folder now.

Create new employee-controller in server/controllers

Create employee routes

$ mv server/routes/users.js server/routes/employees.js

Quick test on created routes using curl

$ curl -i http://localhost:3000/employees
$ curl -i http://localhost:3000/employees/view/1
$ curl -i http://localhost:3000/employees/create
$ curl -i http://localhost:3000/employees/edit/1
$ curl -i -X POST http://localhost:3000/employees/save
$ curl -i -X POST http://localhost:3000/employees/update/1
$ curl -i -X POST http://localhost:3000/employees/delete/1

Wednesday, January 10, 2018

Logging with logback


It is important to add proper loggers into our application because when something bad happens, it is the way to gather related information.

There are multiple libraries for logging in Java like JUL, log4j, logback. Spring uses logback as the default library.
When we add any starter dependency that will add starter-logging as well since it depends transitively on the logging starter.


Complete source code, todoapp
$ git clone
$ cd todoapp
$ gradle clean bootrun

Log Levels

Log messages can be used in several levels.
If WARN is enabled, then WARN, ERROR, FATAL will be logged


Appenders responsible for writing log statements.
  • ConsoleAppender – writes messages to the system console
  • FileAppender – appends messages to a file
  • RollingFileAppender – extends the FileAppender with the ability to roll over log files
  • SMTPAppender – sends log messages in an email, by default only for ERROR messages
  • DBAppender – adds log events to a database
  • SiftingAppender – separates logs based on a run time attribute

Layouts and Encoders

Layouts and encoders transform a log message to the desired output format.
  • The PatternLayout - This layout creates a String from a log message based on a conversion pattern.
  • The HTMLLayout - The HTMLLayout displays log data in an HTML table format, to which you can add custom styles.


When you start a spring boot application you can see something like below in the console which comes from the spring boot default configurations.
2018-01-10 13:52:40.579  INFO 16885 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-01-10 13:52:41.157  INFO 16885 --- [           main] s.b.c.e.t.TomcatEmbeddedServletContainer : Tomcat started on port(s): 8080 (http)

Customize logback

We can modify that using
<!-- file -->


%d – Used to output the date of the logging event.
%thread – outputs the name of the thread that the log message occurred in.
$-5level – outputs the logging level of the log message.
%logger{36} – Outputs the name of the logger
%M – Outputs the method name where the logging request was issued.
%msg – Outputs the application-supplied message
%n – line break


For a simple application this will be enough. But for a large application we need more control over logging. So we need to add logback.xml file which will override default configurations.
To include Spring Boot’s configuration, you can add the following inside the tags.

<include resource="org/springframework/boot/logging/logback/base.xml"></include>

WARNING : Not using additivity="false" will cause the message to be printed out twice due to the root log appender and the class level appender both writing to the log

<!-- logback.xml file -->

Using Spring's profile feature we can create separate configuration between environments.

# = dev

Complete source code, todoapp
Read the full tutorial series, spring-rest

Monday, January 8, 2018

REST tutorial : security with JWT token


This is the last part of our security tutorial.
Complete source code
$ git clone
$ cd todoapp
$ git checkout security

follow the complete tutorial


JSON Web Token is an open standard that defines a compact and self-contained way for securely sharing data between parties using a JSON object.
  • Compact - smaller size
  • Self-contained - payload contains all the required information
Typically contains,
  • Header - consists of the type of the token, hashing algorithm being used
  • Payload - contains the claims.
  • Signature
A token fields are separated by dots like this, xxx.yyy.zzz

Since we already have added token support in our previes tutorial, we only have to modify it to a JWT token.

Sample Header

  "alg": "HS512"

Sample Payload

  "sub": "manjula",
  "userId": "1",
  "role": "ADMIN",
  "iat": 1515401468,
  "exp": 1515405068

Testing with JWT

// requesitng a token
$ curl -i -H "Content-Type: application/json" -X POST -d '{"username":"manjula","password":"password"}' http://localhost:8080/token
    "token": "eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJtYW5qdWxhIiwidXNlcklkIjoiMSIsInJvbGUiOiJBRE1JTiIsImlhdCI6MTUxNTQwMTQ2OCwiZXhwIjoxNTE1NDA1MDY4fQ.fzEAwy3110zhYe-XtBUV2Owsr_20CmnbaQ64jnAKh9eoCC41OWwbNddb1Hi-d7cDXARvuko2ADV88iXBos0UqA"

// request with token
$ curl -i -H "Authorization":"Bearer eyJhbGciOiJIUzUxMiJ9.eyJzdWIiOiJtYW5qdWxhIiwidXNlcklkIjoiMSIsInJvbGUiOiJBRE1JTiIsImlhdCI6MTUxNTQwMTQ2OCwiZXhwIjoxNTE1NDA1MDY4fQ.fzEAwy3110zhYe-XtBUV2Owsr_20CmnbaQ64jnAKh9eoCC41OWwbNddb1Hi-d7cDXARvuko2ADV88iXBos0UqA" http://localhost:8080/todos

Congratulations!!! you have successfully created a secure RESTful API.

Sunday, January 7, 2018

REST tutorial : security


So far we have created a nice looking REST API with Java tech stack. But it is missing a crucial piece. Security!!!

Complete source code, todoapp
$ git clone
$ cd todoapp
$ git checkout security

Spring Security

Spring Security is a powerful and highly customizable authentication and access-control framework. It is the de-facto standard for securing Spring-based applications.

Lets add spring security starter to our project.

$ gradle clean bootrun
$ curl -i http://localhost:8080/todos
You will get a message like this,
{"timestamp":1515304279482,"status":401,"error":"Unauthorized","message":"Full authentication is required to access this resource","path":"/todos"}

With just adding the library Spring has secured our application!

Have a look at the console log. You can see something like this,
Using default security password: fc55e01e-16b7-4344-b567-7d6fe90fcb32

Now try this,
$ curl -i -u user:fc55e01e-16b7-4344-b567-7d6fe90fcb32 http://localhost:8080/todos

We get our secured todos!

Spring boot provides us basic security. We only need to override as necessary.

Using username, password for each request is not an ideal for REST. So let's change this to use token based authentication.


  • Handle unauthenticated request
  • Request a token
  • Request resource with the token
  • Token validation and set security context
  • Make it stateless
  • No form login/ logout
  • Secure with user roles



// request a token
$ curl -i -H "Content-Type: application/json" -X POST -d '{"username":"manjula","password":"password"}' http://localhost:8080/token

// receive token like this

// request with token
$ curl -i -H "Authorization":"winteriscoming" http://localhost:8080/todos

In this tutorial we have secured our REST API with a token. But still we can improve it. Let's do it in next tutorial.