5 ways to initialize lazy relations and when to use them

Lazy loading of relations between entities is a well established best practice in JPA. Its main goal is to retrieve only the requested entities from the database and load the related entities only if needed. That is a great approach, if we only need the requested entities. But it creates additional work and can be the cause of performance problems, if we also need some of the related entities.

Lets have a look at the different ways to trigger the initialization and their specific advantages and disadvantages.

1. Call a method on the mapped relation

Lets start with the most obvious and unfortunately also the most inefficient approach. We use the find method on the EntityManager and call a method on the relation.

	Order order = this.em.find(Order.class, orderId);
	order.getItems().size();

view rawmappedRelation.java hosted with  by GitHub

This code works perfectly fine, is easy to read and often used. So what is the problem with it? Well, you probably know it. This code performs an additional query to initialize the relation. That doesn’t sound like a real problem but lets calculate the number of performed queries in a more real world-ish scenario. Lets say we have an entity with 5 relations which we need to initialize. So we will get 1 + 5 = 6 queries. OK, that are 5 additional queries. That still doesn’t seem like a huge issue. But our application will be used by more than one user in parallel (I hope). Lets say our system has to server 100 parallel users. Then we will get 100 + 5*100 = 600 queries. OK, it should be obvious that this approach provides a working but not a good solution. Sooner or later, the number of additionally performed queries will slow our application down. Therefore we should try to avoid this approach and have a look at some other options.

2. Fetch Join in JPQL

A better option to initialize lazy relations is to use a JPQL query with a fetch join.

	Query q = this.em.createQuery("SELECT o FROM Order o JOIN FETCH o.items i WHERE o.id = :id");
	q.setParameter("id", orderId);
	newOrder = (Order) q.getSingleResult();

view rawfetchJoinJPQL.java hosted with  by GitHub

This tells the entity manager to fetch the selected entity and the relation within the same query. The advantages and disadvantages of this approach are obvious:

The advantage is that everything is fetched within one query. From a performance point of view, this is much better than the first approach.

And the main disadvantage is that we need to write additional code which executes the query. But it gets even worse, if the entity has multiple relations and we need to initialize different relations for different use cases. In this case we need to write a query for every required combination of fetch joined relations. This can become quite messy.

Using fetch joins in JPQL statements can require a huge number of queries, which will make it difficult to maintain the code base. So before we start to write lots of queries, we should think about the number of different fetch join combinations we might need. If the number is low, then this is a good approach to limit the number of performed queries.

3. Fetch Join in Criteria API

OK, this approach is basically the same as the one before. But this time we are using the Criteria API instead of the JPQL query.

	CriteriaBuilder cb = em.getCriteriaBuilder();
	CriteriaQuery q = cb.createQuery(Order.class);
	Root o = q.from(Order.class);
	o.fetch("items", JoinType.INNER);
	q.select(o);
	q.where(cb.equal(o.get("id"), orderId));
	Order order = (Order)this.em.createQuery(q).getSingleResult();

view rawfetchJoinCriteria.java hosted with  by GitHub

The advantages and disadvantages are the same as for the JPQL query with a fetch join. The entity and the relation are retrieved with one query from the database and we need specific code for every combination of relationships. But we often already have lots of use cases specific query code, if we are using the Criteria API. So this might not be a huge issue. If we are already using the Criteria API to build the query, this is a good approach to reduce the amount of performed queries.

4. Named Entity Graph

Named entity graphs are a new feature of JPA 2.1. It can be used to define a graph of entities that shall be queried from the database. The definition of an entity graph is done via annotations and is independent of the query. If you are not familiar with this feature, you can have a look at one of my former blog posts where I covered it in more detail.

	@Entity
	@NamedEntityGraph(name = "graph.Order.items",
	attributeNodes = @NamedAttributeNode("items"))
	public class Order implements Serializable {
	....

view rawdefineNamedGraph.java hosted with  by GitHub

The named entity graph can then be used by the find method of the EntityManager.

	EntityGraph graph = this.em.getEntityGraph("graph.Order.items");
	Map hints = new HashMap();
	hints.put("javax.persistence.fetchgraph", graph);
	Order order = this.em.find(Order.class, orderId, hints);

view rawuseNamedGraph.java hosted with  by GitHub

This is basically an improved version of our first approach. The entity manager will retrieve the defined graph of entities from the database with one query. The only disadvantage is, that we need to annotate a named entity graph for each combination of relations that shall be retrieved within one query. We will need less additional annotations as in our second approach, but it still can become quite messy. Therefore named entity graphs are a great solution, if we only need to define a limited amount of them and reuse them for different use cases. Otherwise the code will become hard to maintain. Bonus: Get my free cheat sheet about the new features in JPA 2.1!

5. Dynamic Entity Graph

The dynamic entity graph is similar to the named entity graph and was also explained in one of the former posts. The only difference is, that the entity graph is defined via a Java API.

	EntityGraph graph = this.em.createEntityGraph(Order.class);
	Subgraph itemGraph = graph.addSubgraph("items");
	Map hints = new HashMap();
	hints.put("javax.persistence.loadgraph", graph);
	Order order = this.em.find(Order.class, orderId, hints);

view rawdynamicGraph.java hosted with  by GitHub

The definition via an API can be an advantage and a disadvantage. If we need lots of use case specific entity graphs, it might be better to define the entity graph within the specific Java code and to not add an additional annotation to the entity. This avoids entities with dozens of annotations. On the other hand, the dynamic entity graph requires more code and an additional method to be reusable.

So I recommend to use dynamic entity graphs, if we need to define a use case specific graph, that will not be reused. If we want to reuse the entity graph, it is easier to annotate a named entity graph.

Conclusion

We had a look at 5 different ways to initialize lazy relations. And as we have seen, each of them has its advantages and disadvantages. So what to remember from this article?

• Initializing a lazy relation via calling a method on a mapped relation causes an additional query. This should be avoided for performance reasons.
• Fetch joins in JPQL statements reduce the number of queries to one but we might need a lot of different queries.
• The Criteria API also supports fetch joins and we need specific code for each combination of relations that shall be initialized.
• Named entity graphs are a good solution, if we will reuse the defined graph in our code.
• Dynamic entity graphs can be the better solution, if we need to define a use case specific graph.

reference from http://www.thoughts-on-java.org/5-ways-to-initialize-lazy-relations-and-when-to-use-them/

Linux 同步系統時間

在 Linux 中，可以利用 ntpdate -s time.stdtime.gov.tw 同步全球標準時間，也可以利用 Cron 去自動同步


root #crontab -e
----------------------------------------
#每 30 分鐘同步一次
*/30 * * * * (/usr/sbin/ntpdate -s time.stdtime.gov.tw; /usr/sbin/hwclock -w) > /dev/null 2>&1
----------------------------------------
root #

Centos 7 static ip

https://www.linode.com/docs/networking/linux-static-ip-configuration

透過 ssh 打開遠端的 Linux Desktop Windows

Linux 的 Desktop 一般都是 X Window, 但要遠端登入此 Window 必須用 VNC 等軟體，再透過 XRDP 這類的軟體做橋接

對於 Server 而言，一般都是用 SSH 登入，但萬一需要用到 Server 上的 Desktop 時，可以利用以下介紹的步驟 (NX Server & Client) ，就可以達到透過 SSH 遠端登入的需求，不用再另外開 Port 3389 給 RDP 用

http://wiki.centos.org/zh-tw/HowTos/FreeNX

List cron job for all user in cent os

for user in $(cut -f1 -d: /etc/passwd); do crontab -u $user -l; done

PPPTD for firewall routing

iptables -t nat -A POSTROUTING -j SNAT --to-source [VPS's IP]
iptables -A FORWARD -i ppp0 -o eth0 -j ACCEPT

Rsync 的用法

Copy from http://www.tecmint.com/rsync-local-remote-file-synchronization-commands/

Rsync (Remote Sync) is a most commonly used command for copying and synchronizingfiles and directories remotely as well as locally in Linux/Unix systems. With the help ofrsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.

Rsync Local and Remote File Synchronization

This article explains 10 basic and advanced usage of the rsync command to transfer your files remotely and locally in Linux based machines. You don’t need to be root user to runrsync command.

Some advantages and features of Rsync command

1. It efficiently copies and sync files to or from a remote system.
2. Supports copying links, devices, owners, groups and permissions.
3. It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
4. Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.

Basic syntax of rsync command

# rsync options source destination

Some common options used with rsync commands

1. -v : verbose
2. -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
3. -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
4. -z : compress file data
5. -h : human-readable, output numbers in a human-readable format

Install rsync in your Linux machine

We can install rsync package with the help of following command.

# yum install rsync (On Red Hat based systems)
# apt-get install rsync (On Debian based systems)

1. Copy/Sync Files and Directory Locally

Copy/Sync a File on a Local Computer

This following command will sync a single file on a local machine from one location to another location. Here in this example, a file name backup.tar needs to be copied or synced to/tmp/backups/ folder.

[root@tecmint]# rsync -zvh backup.tar /tmp/backups/

created directory /tmp/backups

backup.tar

sent 14.71M bytes  received 31 bytes  3.27M bytes/sec

total size is 16.18M  speedup is 1.10

In above example, you can see that if the destination is not already exists rsync will create a directory automatically for destination.

Copy/Sync a Directory on Local Computer

The following command will transfer or sync all the files of from one directory to a different directory in the same machine. Here in this example, /root/rpmpkgs contains some rpm package files and you want that directory to be copied inside /tmp/backups/ folder.

[root@tecmint]# rsync -avzh /root/rpmpkgs /tmp/backups/

sending incremental file list

rpmpkgs/

rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm

rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm

rpmpkgs/nagios-3.5.0.tar.gz

rpmpkgs/nagios-plugins-1.4.16.tar.gz

sent 4.99M bytes  received 92 bytes  3.33M bytes/sec

total size is 4.99M  speedup is 1.00

2. Copy/Sync Files and Directory to or From a Server

Copy a Directory from Local Server to a Remote Server

This command will sync a directory from a local machine to a remote machine. For example: There is a folder in your local computer “rpmpkgs” which contains some RPM packages and you want that local directory’s content send to a remote server, you can use following command.

[root@tecmint]$ rsync -avz rpmpkgs/ root@192.168.0.101:/home/

root@192.168.0.101's password:

sending incremental file list

./

httpd-2.2.3-82.el5.centos.i386.rpm

mod_ssl-2.2.3-82.el5.centos.i386.rpm

nagios-3.5.0.tar.gz

nagios-plugins-1.4.16.tar.gz

sent 4993369 bytes  received 91 bytes  399476.80 bytes/sec

total size is 4991313  speedup is 1.00

Copy/Sync a Remote Directory to a Local Machine

This command will help you sync a remote directory to a local directory. Here in this example, a directory /home/tarunika/rpmpkgs which is on a remote server is being copied in your local computer in /tmp/myrpms.

[root@tecmint]# rsync -avzh root@192.168.0.100:/home/tarunika/rpmpkgs /tmp/myrpms

root@192.168.0.100's password:

receiving incremental file list

created directory /tmp/myrpms

rpmpkgs/

rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm

rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm

rpmpkgs/nagios-3.5.0.tar.gz

rpmpkgs/nagios-plugins-1.4.16.tar.gz

sent 91 bytes  received 4.99M bytes  322.16K bytes/sec

total size is 4.99M  speedup is 1.00

3. Rsync Over SSH

With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet.

Also when we use rsync we need to provide the user/root password to accomplish that particular task, so using SSH option will send your logins in an encrypted manner so that yourpassword will be safe.

Copy a File from a Remote Server to a Local Server with SSH

To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using “ssh” with “-e” option and perform data transfer.

[root@tecmint]# rsync -avzhe ssh root@192.168.0.100:/root/install.log /tmp/

root@192.168.0.100's password:

receiving incremental file list

install.log

sent 30 bytes  received 8.12K bytes  1.48K bytes/sec

total size is 30.74K  speedup is 3.77

Copy a File from a Local Server to a Remote Server with SSH

[root@tecmint]# rsync -avzhe ssh backup.tar root@192.168.0.100:/backups/

root@192.168.0.100's password:

sending incremental file list

backup.tar

sent 14.71M bytes  received 31 bytes  1.28M bytes/sec

total size is 16.18M  speedup is 1.10

4. Show Progress While Transferring Data with rsync

To show the progress while transferring the data from one machine to a different machine, we can use ‘–progress’ option for it. It displays the files and the time remaining to complete the transfer.

[root@tecmint]# rsync -avzhe ssh --progress /home/rpmpkgs root@192.168.0.100:/root/rpmpkgs

root@192.168.0.100's password:

sending incremental file list

created directory /root/rpmpkgs

rpmpkgs/

rpmpkgs/httpd-2.2.3-82.el5.centos.i386.rpm

           1.02M 100%        2.72MB/s        0:00:00 (xfer#1, to-check=3/5)

rpmpkgs/mod_ssl-2.2.3-82.el5.centos.i386.rpm

          99.04K 100%  241.19kB/s        0:00:00 (xfer#2, to-check=2/5)

rpmpkgs/nagios-3.5.0.tar.gz

           1.79M 100%        1.56MB/s        0:00:01 (xfer#3, to-check=1/5)

rpmpkgs/nagios-plugins-1.4.16.tar.gz

           2.09M 100%        1.47MB/s        0:00:01 (xfer#4, to-check=0/5)

sent 4.99M bytes  received 92 bytes  475.56K bytes/sec

total size is 4.99M  speedup is 1.00

5. Use of –include and –exclude Options

These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.

Here in this example, rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory.

[root@tecmint]# rsync -avze ssh --include 'R*' --exclude '*' root@192.168.0.101:/var/lib/rpm/ /root/rpm

root@192.168.0.101's password:

receiving incremental file list

created directory /root/rpm

./

Requirename

Requireversion

sent 67 bytes  received 167289 bytes  7438.04 bytes/sec

total size is 434176  speedup is 2.59

6. Use of –delete Option

If a file or directory not exist at the source, but already exists at the destination, you might want to delete that existing file/directory at the target while syncing .

We can use ‘–delete‘ option to delete files that are not there in source directory.

Source and target are in sync. Now creating new file test.txt at the target.

[root@tecmint]# touch test.txt
[root@tecmint]# rsync -avz --delete root@192.168.0.100:/var/lib/rpm/ .
Password:
receiving file list ... done
deleting test.txt
./
sent 26 bytes  received 390 bytes  48.94 bytes/sec
total size is 45305958  speedup is 108908.55

Target has the new file called test.txt, when synchronize with the source with ‘–delete‘ option, it removed the file test.txt.

7. Set the Max Size of Files to be Transferred

You can specify the Max file size to be transferred or sync. You can do it with “–max-size” option. Here in this example, Max file size is 200k, so this command will transfer only those files which are equal or smaller than 200k.

[root@tecmint]# rsync -avzhe ssh --max-size='200k' /var/lib/rpm/ root@192.168.0.100:/root/tmprpm

root@192.168.0.100's password:

sending incremental file list

created directory /root/tmprpm

./

Conflictname

Group

Installtid

Name

Provideversion

Pubkeys

Requireversion

Sha1header

Sigmd5

Triggername

__db.001

sent 189.79K bytes  received 224 bytes  13.10K bytes/sec

total size is 38.08M  speedup is 200.43

8. Automatically Delete source Files after successful Transfer

Now, suppose you have a main web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server.

So, will you wait for transfer to complete and then delete those local backup file manually? Of Course NO. This automatic deletion can be done using ‘–remove-source-files‘ option.

[root@tecmint]# rsync --remove-source-files -zvh backup.tar /tmp/backups/

backup.tar

sent 14.71M bytes  received 31 bytes  4.20M bytes/sec

total size is 16.18M  speedup is 1.10

[root@tecmint]# ll backup.tar

ls: backup.tar: No such file or directory

9. Do a Dry Run with rsync

If you are a newbie and using rsync and don’t know what exactly your command going do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.

Use of this option will not make any changes only do a dry run of the command and shows the output of the command, if the output shows exactly same you want to do then you can remove ‘–dry-run‘ option from your command and run on the terminal.

root@tecmint]# rsync --dry-run --remove-source-files -zvh backup.tar /tmp/backups/

backup.tar

sent 35 bytes  received 15 bytes  100.00 bytes/sec

total size is 16.18M  speedup is 323584.00 (DRY RUN)

10. Set Bandwidth Limit and Transfer File

You can set the bandwidth limit while transferring data from one machine to another machine with the the help of ‘–bwlimit‘ option. This options helps us to limit I/O bandwidth.

[root@tecmint]# rsync --bwlimit=100 -avzhe ssh  /var/lib/rpm/  root@192.168.0.100:/root/tmprpm/
root@192.168.0.100's password:
sending incremental file list
sent 324 bytes  received 12 bytes  61.09 bytes/sec
total size is 38.08M  speedup is 113347.05

Also, by default rsync syncs changed blocks and bytes only, if you want explicitly want to sync whole file then you use ‘-W‘ option with it.

[root@tecmint]# rsync -zvhW backup.tar /tmp/backups/backup.tar
backup.tar
sent 14.71M bytes  received 31 bytes  3.27M bytes/sec
total size is 16.18M  speedup is 1.10

That’s all with rsync now, you can see man pages for more options. Stay connected withTecmint for more exciting and interesting tutorials in future. Do leave your comments andsuggestions.