JPA Pitfalls (11): Iteration / Sorting in Memory or in DB

This blog article is part of a “series of blog articles” about common pitfalls using JPA and ways to avoid them. In this article, we describe a potential problem when filtering in memory instead of DB.

When selecting specific entity instances, it is preferable to push the filter to the database.

The following code retrieves all entities from the database and filters the specific employees in memory:

List<Employee> emps = // Retrieve all employees from DB
List<Employee> berlinEmps = new ArrayList();
for (Employee e : emps) {
  Address a = e.getAddress();
  if ((a != null) && "Berlin".equals(a.getCity())) {
    berlinEmps.add(e);
  }
}

The downside of the above code is that it oads all the employees into memory including those that are not filtered. Inside of a transaction the entity manager caches all loaded entities, so the above code might run into an OutOfMemoryException. Furthermore, databases support efficient filtering mechanism (e.g. using a database index), so it is very likely that the following code executing a query to filter the employees results in better performance:

String queryText = 
    "SELECT e FROM Employee e JOIN e.address a " + 
    "WHERE a.city = 'Berlin'";
List<Employee> berlinEmps = 
    em.createQuery(queryText).getResultList();

There is a similar issue with sorting entities. In most cases it is better to sort the entities in the database using an ORDER BY JPQL query:

String queryText = 
    "SELECT e FROM Employee e JOIN e.address a " + 
    "WHERE a.city = 'Berlin' " + 
    "ORDER BY e.lastname, e.firstname";
List<Employee> berlinEmps = 
    em.createQuery(queryText).getResultList();

Recommendation: Push filtering and sorting of entities to the database. 

You can find an example of this pitfall and its solution in the classes IterationExperiment and OrderingInDBExperiment in this project: https://github.com/akquinet/jpapitfalls.git.