Saturday, December 29, 2018

Docker - Basic Commands

Greetings!

Following are frequently used Docker commands.

  1. Check the docker version.
    
    docker -version
    
    

  2. Pull image from docker repository.
    
    docker pull <image_name:tag>
    docker pull nginx
    
    

  3. List local images.
    
    docker images
    
    

  4. Create a container from docker image.
    
    docker run
    docker run -it -d nginx
    docker run --name my-nginx -it -d nginx
    
    

  5. List running containers.
    
    docker ps
    docker ps -a
    
    

  6. Access the running container.
    
    docker exec
    docker exec -it ubuntu bash
    
    

  7. Stop running container.
    
    docker stop 
    
    

  8. Kill running container.
    
    docker kill 
    
    

  9. Delete a container.
    
    docker rm <container_id>
    
    

  10. Delete an image from local storage.
    
    docker rmi <image_id>
    
    

  11. Build a docker image from a given Docker file.
    
    docker build .
    
    


Tuesday, November 13, 2018

Java 8 - Streams - flatMap()

Greetings!

With stream API it is easy to iterate over a collection and do whatever operation we like to do. Let's say we have a number list which we would like to print power of 2 of each.


List<Integer> attempt1 = Arrays.asList(1, 2, 3, 4, 5);
attempt1.stream().map(x -> x * x).forEach(System.out::println);

// 1, 4, 9, 16, 25


What if we have a List of List? let's try that out.


List<List<Integer>> attempt2 = Arrays.asList(Arrays.asList(1, 2, 3), Arrays.asList(2, 5));
Stream<List<Integer>> attempt2Stream = attempt2.stream();
attempt2Stream.map(x -> x * x).forEach(System.out::println); // compile error
attempt2Stream.map(x -> x.stream().map(y -> y * y)).forEach(System.out::println);

// java.util.stream.ReferencePipeline$3@4c873330, java.util.stream.ReferencePipeline$3@119d7047


As you can see Stream.map() doesn't give us the expected result for the type Stream<List<Integer>>.

This is because map() operation is the type of Stream<T> and in this case it is like a stream of stream. Not only map() operation any other operation will not give us the desired result.

This is where the Stream.flatMap() is helpful.

Stream.flatMap()

Intermediate operation. Like map() operation this accepts the mapping function and apply it to all the elements in the stream and produce a new stream as the result. Difference is this can handle any type of stream. It Flattens the input T argument and merge all the elements into new stream of type R.


<R> Stream<R> flatMap(Function<? super T,? extends Stream<? extends R>> mapper)


This can understand with few examples.


Stream<DataType[]>>
Stream<Set<DataType>>
Stream<List<DataType>>
Stream<Map<KEY, DataType>


This all can be easily mapped to Stream<DataType> with flatMa(). After flattening we can do any other operation.

Array - Stream<DataType[]>


import java.util.Arrays;
import java.util.stream.Stream;

public class FlatMapArray {

    public static void main(String[] args) {
        Integer[][] array = { { 1, 2, 3 }, { 4, 5 } };
        Stream<Integer[]> stream = Arrays.stream(array);

//        stream.flatMap(subarray -> Arrays.stream(subarray)).map(x -> x * x).forEach(System.out::println);
        stream.flatMap(Arrays::stream).map(x -> x * x).forEach(System.out::println);
    }

}

// 1, 4, 9, 16, 25


List - Stream<List<DataType>>


import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.stream.Stream;

public class FlatMapList {

    public static void main(String[] args) {
        List<List<Integer>> lists = Arrays.asList(Arrays.asList(1, 2, 3), Arrays.asList(2, 5));
        Stream<List<Integer>> stream = lists.stream();

//        stream.flatMap(list -> list.stream()).distinct().map(x -> x * x).forEach(System.out::println);
        stream.flatMap(Collection::stream).distinct().map(x -> x * x).forEach(System.out::println);
    }

}

// 1, 4, 9, 25


Map - Stream<Map<Key, Datatype>>


import java.util.HashMap;
import java.util.Map;
import java.util.stream.Stream;

public class FlatMapMap {

    public static void main(String[] args) {
        Map<Integer, String> map = new HashMap<>();
        map.put(1, "one");
        map.put(2, "two");
        map.put(3, "three");
        
        Stream<Map<Integer, String>> streamMap = Stream.of(map);
        
        streamMap.flatMap(m -> m.values().stream()).map(String::toUpperCase).forEach(System.out::println);
    }

}

// ONE, TWO, THREE


Set - Stream<Set<DataType>>


import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.Stream;

public class FlatMapSet {

    public static void main(String[] args) {
        Set<String> set = new HashSet<>();
        set.add("one");
        set.add("two");
        set.add("three");
        
        Stream<Set<String>> streamSet = Stream.of(set);
        
        streamSet.flatMap(Collection::stream).map(String::toUpperCase).forEach(System.out::println);
    }

}

// ONE, TWO, THREE


In summary these operations behave like this; (DataType is String here)

Stream<String> flatMap(Function<String[], Stream<String>> mapper)

Stream<String> flatMap(Function<List<String>, Stream<String>> mapper)

Stream<String> flatMap(Function<Set<String>, Stream<String>> mapper)

Stream<String> flatMap(Function<Map<Integer, String>, Stream<String>> mapper)




Java 8 - Streams - reduce()

Greetings!

Stream API provides several terminal operations for common tasks like count, min, max, sum and average.  These operations return a single value and do specific task. Stream.collect() operation is also a terminal operations but it return a Collection.
reduce() is a more-general purpose operations which can be used to combine the content of stream.

  • These terminal operations are called reduction operations.

Stream.reduce()

This method mainly has two forms.

  • With initial value.
  • Without initial value.

T reduce(T identity, BinaryOperator<T> accumulator);

Optional<T> reduce(BinaryOperator<T> accumulator);


reduce uses BinaryOperator (which extends BiFunction) functional interface which has the form below.

R apply(T t, U u);


How Does It Work

Let's consider below sum operation.

List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5);
int result = 0;
for (Integer number : numbers) {
    result = result + number;
}
System.out.println(result);


We have a result variable with an initial value and during iteration we add current iteration value to result and assign the calculated value back to result.
Now, this is a reduce operation which fulfills reduce() methods contract.
So, this same operation can be written using reduce like this;

int result8 = numbers.stream().reduce(0, (a, b) -> a + b);
System.out.println(result8);

a - result, b - iteration value
When we are reducing a Collection using iteration which depends on the last calculated value, it is good time to use reduce().

More examples;

import java.util.Arrays;
import java.util.List;

public class ReduceApp {

    public static void main(String[] args) {
        // Java 7
        List<Integer> numbers = Arrays.asList(1, 2, 3, 4, 5);
        int result = 0;
        for (Integer number : numbers) {
            result = result + number;
        }
        System.out.println(result);
        
        // Java 8
        int result8 = numbers.stream().reduce(0, (a, b) -> a + b);
        System.out.println(result8);
        
        int multiplication = numbers.stream().reduce(1, (a, b) -> a * b);
        System.out.println(multiplication);
        
        int evenPow = numbers.stream().filter(i -> i % 2 == 0).reduce(0, (a, b) -> a + b * b);
        System.out.println(evenPow);
        
        List<String> target = Arrays.asList("H", "e", "l", "l", "o");
        String reversed = target.stream().reduce("", (a, b) -> b + a);
        System.out.println(reversed);
    }

}





Monday, November 12, 2018

Hibernate - Basic Annotations

Greetings!

  • @Entity - Add in class level. Declares that this class is an entity

@Entity
public class Employee {
    // ...
}


  • @Table - Add in class level. Define table, schema, catalog. Use name property to add the name of the table unless class name is used as the table name.

@Entity
@Table(name = "tbl_employee")
public class Employee {
    // ...
}


  • @Column - Add column properties.

@Column(name = "first_name")
private String firstName;


  • @Id - Declares property as the identifier of this class.

@Id
private Long id;


  • @GeneratedValue - use along with @Id to generate primary key automatically. JPA defines 5 strategies.

@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;


AUTO - select depending on the underlying database.
IDENTITY
SEQUENCE
TABLE
identity copy - identity is copied form another entity.

  • @Version - Adding this will add optimistic locking capability. For an update query Hibernate will automatically add version number validity in where clause. Numeric is preferred but a timestamp can also be used.

@Version
@Column(name="optimistic_lock")
private Integer version;


  • @Temporal - Use to mark date field format like DATE (date only), TIME (time only) or TIMESTAMP (date and time).

@Column(name = "dob")
@Temporal(TemporalType.DATE)
private Date dateOfBirth;


  • @Enumerated - Use to map enum. By default it used enum's ordinal and EnumType can be used to override default behaviour.

@Column(name = "employee_type")
@Enumerated(EnumType.STRING)
private EmployeeType employeeType;


  • @Embedded - Define to use value object inside the entity class.
  • @Embeddable - Define a value object to use in an entity.
  • @Transient - Skip persisting a property.



Hibernate - Value Types

Greetings!

When we design our domain models we can have more important classes and less important classes like Address, String, etc. In other word we have fine-grained object model which means more classes than tables.
For an example User can have an Address. We can create separate class for Address fields and add it as a property in User using composition.


public class User {
    private Long id;
    private String name;
    private Integer age;
    private Address address;
}


Here User is an entity. Name, Age, Address are value types.

  • Entity classes need an identifier.
  • Value type doesn't have an identifier because instances are identified through owning entity.

What is an Entity?

  • Has a database identity.
  • Has it's own life cycle.
  • Object reference is persisted in the database.

What is a Value Type?

  • Doesn't have own identity.
  • Embedded into owning entity.
  • Represent table columns.

How to map?

  • Basic types can be directly mapped using @Column. (String, Integer, etc)
  • Collection value types using collection mapping.
  • Composite value types can be mapped using @Embedable annotation.




Hibernate - Identity

Greetings!

Identity is the fact that being who or what a thing is.

In Java two objects are identical if they are referring to the same memory location.

object1 == object2


Equals means that they have the same value but may not have same memory location.

object1.equals(object2)


In relational databases, objects are identical if they share the same table and have same primary key. This is known as database identity in Java side.

@Id
@GeneratedValue
private Long id;

  • Because @Id on the field, hibernate use field access.
  • If we don't provide @GeneratedValue, JPA provider doesn't provide a key for us. We have to assign ourselves. These are called application-assigned identifiers.

Candidate Key

Column(s) we could use to identify a database row is called candidate keys.

  • Never null.
  • Unique.
  • Never changes.

Natural Key

A key with a business meaning is called a natural key. For an Employee table can have a employee number to represent employee's identity.
Practically, this will have side effects when the system grows.
- composite natural keys are keys combining two or more columns.

Synthetic Key / Surrogate Key

A unique, not null column generated by the database or the application with no business meaning is called surrogate key.
This is the preferred option.




Hibernate - Introduction

Greetings!

Problem in hand

Almost all the applications need to save data. This persistence storage most probably is a database. This storage is independence system which is called data independence. Here comes the problem. How do we convert relational data to Java objects? Sure we can convert one by one which is not convenient due to;
  • It takes lot of time to write code.
  • Difficult to maintain.
  • Need to write all SQL queries.
  • Need to handle transactions.
  • Have to maintain database connections.
  • Not portable.
To solve these kind of problems, Object Relational Mapping frameworks are born. Hibernate is the most popular among them.

In an era of Spring dominating most part of the application, Hibernate is almost hidden. Thanks to Spring Data most of the time we do not have write any SQL query. Spring help us to generate those.

JPA, Hibernate, Spring Data JPA

Java Persistence API is the specification. Hibernate is one of the implementations of JPA. Spring Data provides us another layer to hide JPA by giving magical interfaces to generate SQL.

Don't we have to learn SQL then?

Without knowing anything about SQL, still we can connect to database and save data, read them, etc. Working with databases is a complex topic and difficult to remember. But to get maximum from ORM framework we better know queries, how to optimize them, and so on. Unless we will end up poorly performing system. For an example, for a data set which we can select using single query, hibernate will generate multiple queries which is the famous N+1 problem.

As I mentioned earlier, we do not have to learn all the things about Hibernate either. Practically we are not going to use Hibernate along. Most of the time we are using Spring. We only have to mark our class as a database entity. Spring simplifies datasource management, transaction management, SQL generation for us.

What is important is, we need to learn Hibernate and JPA to map entities.

Hibernate ORM is an object relational mapping tool for Java.



Sunday, November 11, 2018

Java Concurrency - ThreadLocal

Greetings!

Protecting shared, mutable data is difficult. Easiest way to guarantee thread safety is not to share. This technique is called thread confinement.

One way to achieve this is using ThreadLocal class.
This uses internal thread-local variable to store data by the executing thread and provides get() and set() methods to access it.
It internally get the current thread (Thread.currentThread()) and a custom Map to assign data directly in Thread class.
This behaves like Map<Thread, T> object (though this is not how it is done).

ThreadLocal stores per-thread value and provides set(), get() methods to access it.


public class TimeHolder {

    private static ThreadLocal<Long> threadLocal = ThreadLocal.withInitial(() -> System.currentTimeMillis());
    
    public Long get() {
        return threadLocal.get();
    }
    
    public void set() {
        threadLocal.set(System.currentTimeMillis());
    }
    
    public void print() {
        System.out.println(Thread.currentThread().getName() + " : " + get());
    }
    
}


public class ThreadLocalApp {

    public static void main(String[] args) throws InterruptedException {
        TimeHolder timeHolder = new TimeHolder();
        
        timeHolder.print();
        
        Thread t1 = new Thread(() -> timeHolder.print());
        t1.start();
        t1.join();
        
        timeHolder.print();
        
        Thread t2 = new Thread(() -> {
            timeHolder.set();
            timeHolder.print();
        });
        t2.start();
        t2.join();
        
        timeHolder.print();
        
        timeHolder.set();
        timeHolder.print();
    }

}


This gave below output.

main : 1541956810082
Thread-0 : 1541956810085
main : 1541956810082
Thread-1 : 1541956810086
main : 1541956810082
main : 1541956810086


Notice how each thread is having it's own value meaning threads can't see each other's value.



Java Concurrency - Immutable

Greetings!

Problem with shared object is any thread can modify it. What if we have an object type which can not be changed? This is where immutable objects come into play.

Immutable object is an object that once created it's state can not be changed.

We can construct an immutable object by protecting the inner state by not sharing any setter method to change inner state.

How to construct an immutable class?


  • Do not provide setter methods.
  • Make all fields final and private.
  • Make the constructor private and use a factory method to create objects.
  • Do not share mutable objects.
  • If mutable objects need to be shared, share a copy.
  • If the constructor accepts external mutable objects as parameters create a copy of them.
  • Make the class final to avoid sub classing.


public final class Counter {
    
    private final int count;
    
    private Counter(int initialCount) {
        this.count = initialCount;
    }
    
    public static Counter newCounter(int initialCount) {
        return new Counter(initialCount);
    }
    
    public int getCount() {
        return this.count;
    }
    
    public Counter add(int value) {
        return new Counter(this.count + value);
    }

}


Java Built-in Immutable Classes

Java standard API includes many immutable classes. Most famous one is String. Below is a list of few of them.

  • String
  • Integer, Long, Float, Double, Boolean, Short, Char, Byte
  • java.math.BigInteger, java.math.BigDecimal
  • java.io.File
  • java.util.UUID


  • Note that assigned variable is not immutable and hence not thread safe. Other threads can replace the constructed object.






Java Concurrency - Visibility

Greetings!

Synchronization is not only protecting critical sections. It also provide memory visibility. That means, when one thread modify the state other threads should be able to see the changes without any issue.

We should not share objects without proper synchronization. Unless it will lead to stale data which causes;
  • Unexpected exceptions.
  • Infinite loops.
  • Corrupted data.
  • Inaccurate results.

Out of thin air safety

A thread can see some value of shared data which is updated by another thread (even it is stale) is called out-of-thin-air safety.

All threads should see most up-to-date values of shared mutable variables.

Volatile

It is a weaker form of synchronization.
  • Do not re-order.
  • Put in shared memory (not in cpu's cache).
  • Good for simple situations like maintaining a flag, completion flag.
  • Does not guarantee atomicity. Guarantee only visibility.

Publishing

Make an object available outside of its current scope.
  • public variables.
  • non private setters.

Escape

Publish an object when it should not.

Thread confinement

When data is not shared and accessible only form single thread it is called thread confinement.

Non-safe objects within the thread context is still thread-safe.

Java Concurrency - Volatile

Greetings!

volatile is a keyword in Java. When we declare a variable as volatile it doesn't store in CPU cache. Instead it is put in shared memory.
Because of that multiple threads can access the same memory location and get the updated value. Hence this is considered as a weaker synchronization mechanism.
volatile is not suitable for atomic operations. It is best suited for single operations like a flag.


private volatile boolean completed;


  • Store in main memory. never store in thread locally.
  • Guarantee visibility.
  • Not suitable for atomic operations like read-modify-write.
  • No lock involved.
  • Never block other threads.
  • Suitable for operations like having a flag.
  • Can be used if the value doesn't depend on previous value.
  • Have a performance hit because it doesn't store in cache.
  • Weaker form of synchronization.


Saturday, November 10, 2018

Java Concurrency - Intrinsic Lock & Reentrancy

Greetings!

Any Java object can implicitly act as a lock for purposes of synchronization. These built-in locks are intrinsic locks.

Java provides synchronized block as a built-in locking mechanism to enforce atomicity.

Java built in object lock is called Intrinsic Lock.


synchronized (lock) {
    // shared data
}


  • When use in method level, invoking object is the lock.
  • Act as a mutex (only one thread at a time)
  • Lock is acquired by the executing thread before entering a synchronized block.

Reentrancy

Thread owning a lock can again acquire the lock is called reentrant lock. Can claim the lock multiple time without blocking itself.
  • Without reentrant lock, there will be a dead lock.
  • Intrinsic locks are re-entrant.
  • Locks are acquired on a per-thread basis rather than per-invocation basis.

Every shared, mutable state should be guarded by the same lock.



Java Concurrency - join()

Greeting!

Let's see the output of below program first.

public class ThreadJoin {

    public static void main(String[] args) {
        System.out.println("start");

        Thread t1 = new Thread(() -> System.out.println("in t1"));
        Thread t2 = new Thread(() -> System.out.println("in t2"));

        t1.start();
        t2.start();

        System.out.println("end");
    }

}


start
end
in t1
in t2


Notice that start and end print before it prints the messages from the threads. This is because thread are running separately hence main thread executes immediately.
What if we want to print end after executing t1 and t2? We can ask main thread to wait for finishing t1 and t2.
It is like a relay race. Second runner is waiting for the first runner to come and hand over the flag to him.
join - execution thread put on wait until this thread is finished.

public class ThreadJoin {

    public static void main(String[] args) {
        System.out.println("start");

        Thread t1 = new Thread(() -> System.out.println("in t1"));
        Thread t2 = new Thread(() -> System.out.println("in t2"));

        t1.start();
        t2.start();

        try {
            t1.join();
            t2.join();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        System.out.println("end");
    }

}


start
in t1
in t2
end


"main" thread waited for t1 and t2 to finish there tasks before printing "end".

In a case where the thread is dead locking or taking a long time to finish, join also accepts waiting time in milliseconds.



Java Concurrency - Race Conditions - Read-Modify-Write

Greetings!

When two or more threads access shared data at the same time race condition occurred. These operation should be atomic to thread-safe and we call them "compound actions".

  • read-modify-write - read a value, modify it, write back (ex: increment a value).
  • check-then-act - check a condition and act accordingly; common situation is lazy initialization (ex: singleton pattern).

In this blog post we are going to examine read-modify-write race condition.

Let's say we need a counter and we designed below class.


public class Counter {

    private int count = 0;

    public int getAndIncrement() {
        return count++;
    }

    public int get() {
        return count;
    }

}


Can you spot a possible bug here? Well, for a single threaded application this is working fine. But for a multi-threaded application this will give unexpected results.

Problem here is count++ is not an atomic operation.

Let's create few threads and see the results.


import java.util.stream.IntStream;

public class CounterApp {

    public static void main(String[] args) throws InterruptedException {
        Counter counter = new Counter();

        Thread thread1 = new Thread(() -> IntStream.range(0, 100)
                                                    .forEach(i -> counter.getAndIncrement()));
        Thread thread2 = new Thread(() -> IntStream.range(0, 100)
                                                    .forEach(i -> counter.getAndIncrement()));

        thread1.start();
        thread2.start();

        thread1.join();
        thread2.join();

        System.out.println("Count: " + counter.get());
    }

}


Each thread increment 100 times, so I expected 200 as result. Surprise!!! This gave me results like 155, 200, 140, 196, etc. In each run it gives different results.
(If you are always getting 200, try increasing to a higher value)

what is happening here is both threads read and update (write) the counter at the same time but the changes are not visible.

We can fix this using synchronized.


public class Counter {

    private int count = 0;

    public synchronized int getAndIncrement() {
        return count++;
    }

    public synchronized int get() {
        return count;
    }

}



Java Concurrency - How To Create A Thread

Greetings!

It is easy to create a thread in Java using 2 well known methods.
  • extends Thread class.
  • implement Runnable interface.
Then we create a Thread object and start it! It is as simple as that.


public class ExtendThread extends Thread {
    
    @Override
    public void run() {
        System.out.println("hello from extended thread");
    }
    
}


Thread thread = new ExtendThread();
thread.start();



public class ImplmentedRunnable implements Runnable {

    @Override
    public void run() {
        System.out.println("hello from runnable implementation");
    }
    
}



Thread thread = new Thread(new ImplmentedRunnable());
thread.start();


In Java 8;

Thread thread = new Thread(() -> System.out.println("hello from java 8"));
thread.start();



  • It is not a good practice to extend the Thread class.


Thread states.

A thread exist in several states.
New - when we create an instance of Thread class it is in New state.
Running - when we call start()
Suspended -
Blocked -
Terminated -



Java Concurrency - Thread Safety

Greetings!

What is thread safety?

When we have multiple threads accessing piece of code simultaneously without unwanted side effects we can say our code is thread safe. Correctness!!!

Synchronization

Synchronization allows to control the program flow and access to shared data for concurrently executing threads.

It is tempting to think synchronized is the only way to do this in Java but it is the primary mechanism which provides exclusive locking. volatile variable, explicit locks and atomic variables are other types.

We can synchronize using;
  • mutex lock - allow only one thread at a time to execute a specific section of code (critical section !!)
  • read/write lock - permit concurrent reads and exclusive writes to a protected shared resource.
  • condition variable - block threads until a particular condition is true.
  • semaphore - simply a variable which is a count (counting semaphore) or 0/1 switch (binary semaphore).

How to share mutable state between threads?

  • Don't share at all.
  • Make it immutable.
  • Use proper synchronization mechanism.

Tip :- Always try to use good programming techniques - Object oriented practices, encapsulation, immutability.
Stateless objects are always thread-safe!!!
Atomicity - Set of actions is atomic when all execute as a single operation.
Mutex (Mutual Exclusion) - At a given time only one thread can enter its critical section is called mutual exclusion.
Avoid holding the lock for lengthy computations or operations.

Java Concurrency - Introduction

Greetings!

We can sing while having a bath. We can listen to song while jogging. We are multitasking!. Can a computer to do that? How can a computer do that? As a developer how do we do that? 


Process vs Thread

Let's take word processor as an example. In a word processor there are spell checker, auto correct, etc. Which means within Word application there are sub tasks are running behind the scene. And those tasks share the same address space as the process.
We name those sub tasks as Threads and main task as a Process.

Back in old days, a computer had only one CPU. On the other hand computer could handle only one task at a time. It switch between tasks very quickly so that we can't see the difference. (Time Slice)

Modern computers come with multiple processors allowing multiple tasks to run at the same time.

When dealing with multiple threads we have to think;

  • Two threads update a variable simultaneously.
  • One threads depends on another thread.
  • Two threads are waiting for each other.
  • Thread order (who has the highest priority).
  • Will all the threads get chance to execute?.
  • and more...


Benefits of multi-threading

  • Better resource utilization.
  • Fairness
  • Better performance.
  • More responsive

Costs of multi-threading

  • Complex design.
  • CPU overhead to switch between threads.

Issues with multi-threading

In a multi-threaded application will mainly face two kind of problems.
  • Visibility issue - One thread change the data while the other thread reading it unaware of the change.
  • Access issue - Change same shared data.
Eventually these will lead to;
  • liveness failure - stop reacting (deadlock)
  • safety failure - incorrect data produces (race conditions)
  • performance - due to switch between threads

Java Collections - Lists

Greetings!

It is probably Lists are the most used Collections.
  • Can have duplicates.
  • Maintain the element order.
  • Can iterate in added order.

Lists maintain the added order. Hence it can be accessed by given position. List interface have additional methods to fulfill this requirement.


// add, remove in given position
void add(int index, E e)
E get(int index)
E remove(int index)
E set(int index, E e)

// search position by object
int indexOf(Object o)
int lastIndexOf(Object o)

// get a sub list by given positions (toIndex exclusive)
List<E> subList(int fromIndex, int toIndex)

// get listiterator
ListIterator<E> listIterator()

  • subList doesn't return a new list. It returns a view of the existing list by given positions. Because of this changing the original list changes the sub list. But changing the sub list doesn't affect the original list.
  • ConcurrentModificationException can be occurred if the original list is changed and try access the sub list.
  • ListIterator has methods to traverse backward using hasPrevious(), previous()

Additionally, Lists implement RandonAccess interface.

ArrayList

  • Implemented as a re-sizable array.
  • add, remove in middle is slow (to shift index).
  • Can get from any position.

LinkedList

  • Implemented as a double linked list.
  • add, remove is fast. (since only the link is affected)
  • accessing an element is slower. (have to traverse through links)

import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
import java.util.stream.IntStream;

public class ListsApp {

    public static void main(String[] args) {
        // add operation
        long start = System.nanoTime();
        List<String> arrayList = new ArrayList<>();
        IntStream.range(0, 10000).forEach(i -> arrayList.add("Hello"));
        long arrayListAdd = System.nanoTime() - start;

        start = System.nanoTime();
        List<String> linkedList = new LinkedList<>();
        IntStream.range(0, 10000).forEach(i -> linkedList.add("Hello"));
        long linkedListAdd = System.nanoTime() - start;
        
        System.out.println(arrayListAdd - linkedListAdd);
        
        // get operation
        start = System.nanoTime();
        IntStream.range(0, 10000).forEach(i -> arrayList.get(i));
        long arrayListGet = System.nanoTime() - start;
        
        start = System.nanoTime();
        IntStream.range(0, 10000).forEach(i -> linkedList.get(i));
        long linkedListGet = System.nanoTime() - start;
        
        System.out.println(arrayListGet - linkedListGet);
    }

}




Java Collections - Interfaces

Greetings!

There are number of data structures. To handle varies kind of collections, Collection framework has different interfaces.

  • Collection - fundamental interface for most of the collections
  • Set  - no duplicates. no order.
  • List - ordered collection. contain duplicates. can access using position.
  • Queue - typically has first-in-first-out structure.
  • Deque - double-ended queue. can behave both as first-in-first-out and last-in-last-out. can insert and remove from both ends.
  • SortedSet - maintain sort order
  • Map - contains key-value pairs. no duplicates allowed.
  • SortedMap - maintain sorted key order

Collection Interface


Map Interface


Iterator


Other than that, there is RandomAccess interface.


Java Collections - Collection Interface

Greetings!

Collection interface is the base interface for most of the classes in Collection Framework. It mainly contains methods for add, remove and iteration.


public interface Collection<E> {

    boolean add(E element);

    Iterator<E> iterator();

    boolean remove(Object o);

    // ....
}


Other than that this contains a lot of common utility methods.


int size()
boolean isEmpty()
boolean contains(Object obj)
boolean containsAll(Collection<?> c)
boolean equals(Object other)
boolean addAll(Collection<? extends E> from)
boolean remove(Object obj)
boolean removeAll(Collection<?> c)
void clear()
boolean retainAll(Collection<?> c)
Object[] toArray()
<T> T[] toArray(T[] arrayToFill)


Java 8 has introduced new default methods to convert the Collection into a stream.


default Stream<E> stream()
default Stream<E> parallelStream()
default boolean removeIf(Predicate<? super E> filter)


  • There is no concrete implementation of Collection interface. Instead it has several sub interfaces to handle different kind of Collections.
  • Because most of the utility methods are common, there is partially implemented AbstractCollection that other implementation can extend. (this can be removed with Java 8 default methods)



Java Collections - ConcurrentModificationException

Greetings!

When you use a collection and suddenly encounter ConcurrentModificationException it is very confusing. Why are we getting this when we are not using multi-threading?

Actually this doesn't mean you are using multi-threading. This means that the collection in used is modified (add/remove).

For an example let's say we want remove multiples of 3 from an array list.


List<Integer> numbers = IntStream.range(1, 20).boxed().collect(toList());

for (Integer number : numbers) {
    if (number % 3 == 0) {
        numbers.remove(number - 1);
    }
}


This little program will throw java.util.ConcurrentModificationException because we are modifying the collection directly while iterating.

Underline iterator is implemented so that it checks for any modification. If it found any modification it throws new ConcurrentModificationException.

Instead of removing items from the collection, we need to remove it from the iterator.


List<Integer> numbers = IntStream.range(1, 20).boxed().collect(toList());

Iterator<Integer> iterator = numbers.iterator();

while (iterator.hasNext()) {
    if (iterator.next() % 3 == 0) {
        iterator.remove();
    }
}


In Java 8, above while loop has been added to Collection interface as a default method;


List<Integer> numbers = IntStream.range(1, 20).boxed().collect(toList());

numbers.removeIf(number -> number % 3 == 0);




Java Collections - Iterable and Iterator

Greetings!

It is easily misunderstand that Collection interface is the super interface in JCF. Actually, Collection interface extends Iterable interface.


public interface Iterable<T> {

    Iterator<T> iterator();

}



Idea of this is to give a common way traverse the underline collection using the Iterator.


public interface Iterator<E> {

    boolean hasNext();

    E next();

    void remove()

}



Prior to Java 5, for loop was used as below,


for (Iterator iterator = collection.iterator(); iterator.hasNext(); ) {
   iterator.next();
}



With Java 5 foreach loop, iterators are used internally by the compiler and it is simplified.


for (SomeObject object : collection) {

   // ....

}



With Java 8 lambda expressions both Iterable and Iterator interfaces are updated to simplify this even more.


iterator.forEachRemaining(element -> {

   // do something

});


collection.foreach(element -> {

   // do something

});



next() and remove()

When we call next(), iterator jumps to the next element and return it. It is like it cosumes the current element.
remove() method remove an element which is returned by the next(). That mean if we want to remove an item we first need to receive it.

Let's say we need to remove the first number from an given numbers collection;


Iterator<Integer> iterator = numbers.iterator();
iterator.remove();


This throws IllegalStateException. Correct way to do this is first load it.


Iterator<Integer> iterator = numbers.iterator();
iterator.next();
iterator.remove();


To remove an element, you must call next() to jump over to it.

Java Collections - Introduction

Greetings!

We have to use different kind of data structures and algorithms in our code. Luckily for us, Java provide Java Collection Framework (JCF) for that purpose.

Prior to Java 1.2 Java had only few data structures like Vector, Stack, Hashtable, Enumeration.

Java 2 introduced proper collection API to address all data structures issues.
  • Easy to use collections.
  • High performance.
  • Easy to extend.
There is no "one for all" implementation.

Common Interfaces

  • Iterable - provide iteration through a collection. (this is outside of the api)
  • Collection - main interface for the most of the collections. contains most of the core functionalities.
  • Set - can not have duplicate. contains unique elements. doesn't maintain the insertion order.
  • List - maintain the order. can have duplicates.
  • Queue - define first in first out data structure.
  • Map - key, value pairs.

In Collection API, interfaces provide the necessary contract but it doesn't give us an idea about real algorithm used. Multiple implementations have multiple purposes, hence we need to check the implemented class also to get the real idea.

How do they implemented?

  • Array - arrays maintain adjacent positions and implemented directly in hardware. Because it maintains the position removal and insertion at given position is slow. Read at a given point, iteration is fast. (ArrayList, CopyOnWriteArrayList, EnumSet, EnumMap)
  • Linked list - maintain a link to next item (and to previous item). Reading in middle is slow due to this. But insertion and removal is fast. (LinkedList)
  • Hashtable - index by content. no support to access by position. (Set, Map)
  • Trees - organize by content but can store and retrieve in sorted order. (TreeSet, TreeMap)

Friday, November 9, 2018

Java 8 - Streams - How to Create a Stream?

Greetings!

It is easy to guess that we can create a Stream using a collection. Actually it is one of the ways to create a Stream.

Stream from collections


Stream<Employee> stream = employees.stream();


Stream from values

Using static Stream.of() we can create a stream using given values.

Stream<String> countryStream = Stream.of("Sri Lanka", "India", "Pakistan");
Stream<Integer> integerStream = Stream.of(1, 2, 3);
Stream<Employee> employeeStream = Stream.of(new Employee());


Stream from arrays

Arrays class has Arrays.stream() static method to convert given array into a stream.

int[] intArray = { 1, 2, 3, 4, 5 };
IntStream intStream = Arrays.stream(intArray);
String[] stringArray = { "Hello", "World" };
Stream<String> stringStream = Arrays.stream(stringArray);


Stream from files

Files class has static methods to open a file as a stream.

// from a file
try (Stream<String> linesStream = Files.lines(Paths.get("pom.xml"))) {
   linesStream.forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

// print files in current working directory
try (Stream<Path> paths = Files.list(Paths.get(""))) {
    paths.forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}


Stream from numeric ranges


// Inclusive. Up to 100
IntStream.rangeClosed(1, 100).forEach(System.out::println);

// Exclusive. Up to 99
IntStream.range(1, 100).forEach(System.out::println);

// With Steps
IntStream.iterate(0, i -> i + 2).limit(50).forEach(System.out::println);



Java 8 - Streams - Replace Traditional for Loops

Greetings!

It is very common to have numeric iterations like below;


for (int i = 1; i <= 100; i++) {
    // do something
}


Java 8 has static methods for this common task in IntStream and LongStream.

IntStream intStream = IntStream.rangeClosed(1, 100);

// Inclusive. Up to 100: 1, 2, ..., 99, 100
IntStream.rangeClosed(1, 100).forEach(System.out::println);

// Exclusive. Up to 99: 1, 2, ..., 98, 99
IntStream.range(1, 100).forEach(System.out::println);

// With Steps: 1, 2, ..., 98, 100
IntStream.iterate(0, i -> i + 2).limit(50).forEach(System.out::println);

Note that iterate produces an infinite stream.

Java 8 - Streams - Numeric Streams

Greetings!

It is common to have numeric operations min, max, sum in our code. Stream interface have min and max methods but doesn't have sum method because it make no sense to have it.
Even though we can have sum operation using reduce() it has unboxing cost. This is same for min, max operations.
Considering those factors, Stream API provides separate interfaces for primitives. IntStream, LongStream and DoubleStream.

Convert to numeric stream

stream.mapToInt() - return IntStream
stream.mapToLong() - return LongStream
stream.mapToDoulbe() - return DoubleStream


IntStream intStream = stream.mapToInt()


Convert back to stream of objects

numericStream.boxed()


String<Integer> = intStream.boxed()


Optional values

For primitive operations Optional also have corresponding primitive optionals.
OptionalInt, OptionalLong, OptionalDouble

There are corresponding functional interfaces like IntPredicate, IntConsumer, IntUnaryOperator for numeric operations.

Example:-

import java.util.Arrays;
import java.util.List;
import java.util.OptionalDouble;
import java.util.OptionalInt;

public class NumericApp {

    public static void main(String[] args) {
        List<Subject> subjects = Arrays.asList(
                Subject.newInstance("Maths", 95), Subject.newInstance("Enlighs", 98),
                Subject.newInstance("Music", 70), Subject.newInstance("Science", 84),
                Subject.newInstance("History", 92));
        
        int total = subjects.stream().mapToInt(Subject::getMarks).sum();
        OptionalInt min = subjects.stream().mapToInt(Subject::getMarks).min();
        OptionalInt max = subjects.stream().mapToInt(Subject::getMarks).max();
        OptionalDouble average = subjects.stream().mapToInt(Subject::getMarks).average();
        
        System.out.println("total: " + total);
        System.out.println("min: " + min);
        System.out.println("max: " + max);
        System.out.println("average: " + average);
    }

}


public class Subject {

    private String name;
    private Integer marks;

    private Subject(String name, Integer marks) {
        this.name = name;
        this.marks = marks;
    }
    
    public static Subject newInstance(String name, Integer marks) {
        return new Subject(name, marks);
    }

    public String getName() {
        return name;
    }

    public Integer getMarks() {
        return marks;
    }

}



Java 8 - Streams - sum()

Greetings!

Let's say we want to calculate total marks for a student for all subjects.


public class Subject {
    private String name;
    private Integer marks;
}


int totalMarks = subjects.stream().map(Subject::getMarks).sum();


This is not working because stream doesn't have sum() operation. That make sense because stream is generic and it doesn't have any meaning for total (sum) operation for an object.
And also if we use Integer, behind the scene it will covert to int (unboxing). For a large data set this will be a costly operation.
So, to get our desired result we to need to convert our stream into IntStream which offers the sum operation.


int totalMarks = subjects.stream().mapToInt(Subject::getMarks).sum();



  • mapToInt - this intermediate operation returns IntStream



Java 8 - Streams - Terminal Operations

Greetings!

Terminal operations are used to close the stream and generate a result. This is the last operation in a stream pipeline hence eagerly executed. Can return concrete value types like (Integer, String, List, Optional), primitive value (int, long) or void.

Ex:-
forEach, collect, count, anyMatch, nonMatch, allMatch, findAny, findFirst, reduce, min, max, sum are terminal operations.

Let's use employee data set for all operations.
(Complete code is here)


public class Employee {
    
    enum Gender { MALE, FEMALE };
    
    private String name;
    private int age;
    private Gender gender;
    
    private Employee(String name, int age, Gender gender) {
        this.name = name;
        this.age = age;
        this.gender = gender;
    }
    
    public static Employee newInstance(String name, int age, Gender gender) {
        return new Employee(name, age, gender);
    }

    // getters

}



    private static List<Employee> employees = Arrays.asList(
            Employee.newInstance("Jon", 22, Gender.MALE),
            Employee.newInstance("Arya", 16, Gender.FEMALE),
            Employee.newInstance("Tyrion", 30, Gender.MALE),
            Employee.newInstance("Ghost", 5, Gender.MALE),
            Employee.newInstance("Joffery", 20, Gender.MALE),
            Employee.newInstance("Hound", 35, Gender.MALE),
            Employee.newInstance("Danny", 23, Gender.FEMALE)
    );


forEach()

Perform an action for each element in stream. Return type is void.


employees.stream()
            .filter(employee -> employee.getAge() > 20)
            .forEach(System.out::println);


collect()

Reduce the stream into another container such as collection.


employees.stream()
            .filter(employee -> employee.getAge() > 20)
            .collect(Collectors.toList());


count()

Return the total number of elements in the stream.


long count = employees.stream().count();

System.out.println(count);

long femaleCount = employees.stream()
                             .filter(employee -> Gender.FEMALE.equals(employee.getGender()))
                             .count();

System.out.println(femaleCount);


min()

This is a special reduction operation. This returns the minimum value of the stream by given comparator. This return an Optional value.


Optional<Employee> min = employees.stream()
          .min((e1, e2) -> e1.getAge() - e2.getAge());

System.out.println(min);


max()

Same as min operation, this return maximum value.


Optional<Employee> max = employees.stream()
          .max((e1, e2) -> e1.getAge() - e2.getAge());

System.out.println(max);


allMatch()

Check whether all elements in the stream match the given condition.


boolean allMatch = employees.stream()
                            .allMatch(employee -> Gender.MALE.equals(employee.getGender()));
System.out.println("Are all employees male ? " + allMatch);


anyMatch()

Check whether any of the elements in stream matches the given condition.


boolean anyMatch = employees.stream()
                            .anyMatch(employee -> Gender.FEMALE.equals(employee.getGender()));
System.out.println("Is there any female ? " + anyMatch);


nonMatch()

Return true if no element in stream matches the given condition.


boolean noneMatch = employees.stream()
                            .noneMatch(employee -> employee.getAge() > 50);
System.out.println("Is there no one above 50 ? " + noneMatch);


findAny()

As the name suggest this returns any element from the stream.


Optional<Employee> any = employees.stream().findAny();
System.out.println(any);

Optional<Employee> anyFemale = employees.stream()
        .filter(employee -> Gender.FEMALE.equals(employee.getGender()))
        .findAny();
System.out.println(anyFemale);


findFirst()

This returns the first element in the stream.


Optional<Employee> first = employees.stream().findFirst();
System.out.println(first);

Optional<Employee> firstFemale = employees.stream()
                                            .filter(employee -> Gender.FEMALE.equals(employee.getGender()))
                                            .findFirst();
System.out.println(firstFemale); 



Java 8 - Streams - Intermediate Operations

Greetings!

Intermediate operations are used to transform the stream into another. These operations can be chained and merge internally as necessary. Lazily process and process only when a terminal operations is called.

Ex:-
map, flatMap, filter, distinct, limit, skip, sorted are intermediate operations

  • Note that forEach is a terminal operation which close the stream.

map()

Convert the iteration element into another.


public static void main(String[] args) {
    List<String> countries = Arrays.asList("Sri Lanka", "India", "Pakistan", "England", "Australia");
    countries.stream().map(String::toUpperCase).forEach(System.out::println);
    countries.stream().map(country -> {
        // create any object and return
        int length = country.length();
        return "Length of ".concat(country).concat(" string is ").concat(String.valueOf(length));
    }).forEach(System.out::println);
}



filter()

Filter the stream elements by given condition.


public static void main(String[] args) {
    List<String> countries = Arrays.asList(
            "Sri Lanka", "India", 
            "Pakistan", "England", 
            "Australia", "South Africa");
    // filter country names started with S
    countries.stream().filter(country -> country.startsWith("S")).forEach(System.out::println);
    
    // filter country names length greater than 5
    countries.stream().filter(country -> country.length() > 5).forEach(System.out::println);
}



distinct()

Select distinct values from the stream.


public static void main(String[] args) {
    List<String> countries = Arrays.asList(
            "Sri Lanka", "India", 
            "Pakistan", "England", "Sri Lanka",
            "Australia", "South Africa", "Pakistan");

    countries.stream().distinct().forEach(System.out::println);
}



limit()

Limit the stream by given value.


public static void main(String[] args) {
    List<String> countries = Arrays.asList(
            "Sri Lanka", "India", 
            "Pakistan", "England",
            "Australia", "South Africa");

    // select only first 3 countries
    countries.stream().limit(3).forEach(System.out::println);
}


skip()

Skip n number of values from the stream.


public static void main(String[] args) {
    List<String> countries = Arrays.asList(
            "Sri Lanka", "India", 
            "Pakistan", "England",
            "Australia", "South Africa");

    // skip first 3 countries
    countries.stream().skip(3).forEach(System.out::println);
}


sorted()

Sort the elements in the stream by given condition.


public static void main(String[] args) {
    List<String> countries = Arrays.asList(
            "Sri Lanka", "India", 
            "Pakistan", "England",
            "Australia", "South Africa");

    // sort by natural order
    countries.stream().sorted().forEach(System.out::println);
    
    // sort by string length
    countries.stream().sorted((first, second) -> Integer.compare(first.length(), second.length()))
                        .forEach(System.out::println);
}




Java 8 - Streams - Introduction

Greetings!

Let's have a look at below code.


List<String> countries = Arrays.asList("Sri Lanka", "India", "Pakistan", "England", "Australia");

for (String country : countries) {
    if (country.length() > 7) {
        String upperCase = country.toUpperCase();
        System.out.println(upperCase);   
    }
}


This code has several problems.
  • Process sequentially. Difficult to process parallel.
  • Complex codes.
  • Instruct how to do it, not what to do.
  • External loop. (developer has to create the loop).

To overcome these kind of problems and get the maximum from multi-core processors Java 8 introduces Stream API.

The same code can be re-written using stream as below.


countries.stream()
        .filter(country -> country.length() > 7)
        .map(country -> country.toUpperCase())
        .forEach(System.out::println);


As you can see it works like builder pattern. We chain several operations using a dot.
We can think of this as a sql query. Instead of how to it, we are instructing what to do.
We didn't provide a loop here, instead it uses internal iteration.

Stream operations have 3 stages.
  • Create stream using an array, collection.
  • Transform stream using intermediate operations.
  • Reduce the result using terminal operations.

Intermediate Operations

Intermediate operations return another stream which helps to connect multiple operations.. These operations doesn't process as in the order we provide. Instead it merge possible operations into a single operation. Another important point is these operations process lazily. Operation doesn't invoke until a terminal operation is called.
Ex:- map, flatMap, filter, distinct, limit

Terminal Operations

These operations end the stream and produce a result. Result is not a stream. It can be void, a List, Integer, etc.
Ex: forEach, collect, reduce, count, max, min

Java 8 - Constructor References

Greetings!

This is same as the method references, except the method name is new.

Class::new


When we have multiple constructors which constructor will be selected? It depends on the context.


(firstName) -> new User(firstName) => User::new
(firstName, lastName) -> new User(firstName, lastName) => User::new


This can also be used with arrays with exact one parameter.

n -> new int[n]
int[]::new


This helps to create arrays with generic types.

Object[] array = stream.toArray();
User[] users = stream.toArray(User[]::new);


Java 8 - Method References

Greetings!

In Java we have object references. We pass objects around. What if we can pass reference to a method?
In Java 8 if we are executing only one method we can do this using lambda expressions.

For an example these are equal.   
     

(x) -> System.out.print(x)
System.out::print


As you can see, :: separates the name of the method from the class/ object. There are 3 ways we can use method references.
  1. object::method - object.method(parameters)
  2. Class::staticMethod - Class.staticMethod(parameters)
  3. Class::instanceMethod - firstParameter.method(parameters)
In first 2 cases, parameters become the methods parameters. As you can see in System.out::print. If we have more than 1 parameters it is still same,

(x, y) -> Math.pow(x, y)
Math::pow


Third case is different. First parameter becomes the target object of the method and rest become the parameters. For example,

// (object, arguments) -> object.method(arguments)
name -> name.toUpperCase()
String::toUpperCase


We can use this and super also the same way;

this::instanceMethod
super::instanceMethod


inner class => lambda expression => method reference