Announcing Vendure v1.6

v1.6.0 Changelog

Upgrading from v1.x.x

This minor release contains no breaking schema or GraphQL API changes, so updating should be a matter of changing all @vendure/... dependencies in your package.json file to 1.6.0.

Also see the Updating Vendure guide for more information.

Performance

Vendure is fast. Unlike some other frameworks, it generally does not require carefully-tuned, beefy servers with layers of caching to deliver good results.

Having said that, some users with particularly large stores (tens or hundreds of thousands of products, customers & orders) were reporting certain queries performing poorly - notably when querying lists of items. I decided to do a deep-dive into this topic and created the Core performance improvements issue to collect all relevant information and record research results & benchmarks.

After a week of intensive work, I'm happy to report a 4 - 10x performance improvement in our benchmarks for list queries. Our benchmarks used a modest 10k ProductVariants and 10k Orders, so if your store is bigger than that the performance boost should be even more pronounced!

Achieving these improvements came down to three techniques:

List query optimization

Vendure's list queries are powered by the ListQueryBuilder, a powerful utility for constructing SQL queries with support for sorting, filtering, pagination and more. The main performance issue was caused by the way the ListQueryBuilder was joining relations, sometimes resulting in a huge amount of data being selected in a single query.

It was found that a much more performant way to handle this is to execute multiple queries in parallel, each query joining only a single relation, and then reconcile all the data at the end. In our benchmarks this optimization yielded up to a 4x performance increase for list queries. Indeed, this approach has even been built in to the latest version of TypeORM as the relationLoadStrategy, and is the recommended way to load large amounts of data.

For a more in-depth explanation of the mechanics behind this optimization, see this answer to "JOIN queries vs multiple queries" on StackOverflow.

GraphQL Lookaheads

Another source of performance issues was that we would often join relations that we didn't even need. For example, when fetching a list of Orders, the OrderService.findAll() method would default to always joining the related OrderLine, OrderItem, ProductVariant, Channel and ShippingLine relations. But what if our query only looked like this?

For the above query, we don't need to join any relations! The next idea I explored was to leverage the ability to examine the incoming GraphQL operation to see what fields are actually being requested, and then only join those relations that are strictly necessary to service that operation. It turns out that this approach has a name - GraphQL lookaheads. There's an in-depth article about it over on the Zalando engineering blog.

In our case, implementing lookaheads consists of two parts:

1. determining which relations are required by the API operation and

2. implementing an API to pass this information through to the data layer

The first point is handled with the new Relations decorator. This decorator allows us to inject a parameter into our resolver functions which exposes an array of required relations.

Secondly, this array can then be passed through to the service layer. Most of the "findOne" and "findAll" methods of our built-in services now feature an additional parameter which allows you to specify exactly which relations should be joined when querying an entity or list of entities.

How does that look in practice?

In the above example, given the following query:

then the value of relations will be ['customer', 'lines', 'lines.items'].

The 'customer' comes from the fact that the query is nesting the “customer” object, and the 'lines' & 'lines.items' are taken from the Order entity’s totalQuantity property, which uses the Calculated decorator and defines those relations as dependencies for deriving the calculated value.

The use of lookaheads yielded an additional 2.8x performance increase in the best case (where the query requires no relations to be joined). This approach should also theoretically improve memory usage and database load, although these aspects were not benchmarked.

Improved use of indexes

This issue brought to my attention that in Postgres, indexes are not automatically created on foreign-key relations, unlike in MySQL. This means that with very large amounts of data, join operations result in very inefficient sequential scans of entire tables of data.

The solution is to explicity add indexes on the related entites. Simple! The problem is, doing so will cause a breaking change in the schema - something we must avoid until the next major version (v2.0)!

The solution is a new config option, entityOptions.metadataModifiers which allows us to directly manipulate the TypeORM metadata of any entity. This feature, by the way, has potential use-cases beyond solving this indexing issue! But here's how you can use it to get even better performance out of your Postgres database:

Note that after adding this modifier, you'll need to run a migration.

Collection improvements

Collections in Vendure allow you to group ProductVariants together, typically to model a category hierarchy for navigation purposes. The way ProductVariants get added to a Collection is via CollectionFilters. A filter is basically a function which adds some WHERE clauses to an SQL select query - the results of which determine which ProductVariants make it into the Collection.

This is a maximally-flexible approach to populating Collections - the Collection is "live" and gets updated automatically whenever products change.

The downside to this approach is that it is difficult to know exactly what ProductVariants will end up in your Collection as you configure the filters. You just have to save changes and then hope you got it right!

Live Collection previews

With Vendure v1.6, you can live-preview the contents of your Collection as you configure your filters, in real time! This takes all the guesswork out of constructing your filters, and ensures you have the perfect configuration set up before hitting "save".

Filter combinations

Collection can have multiple filters which express complex conditions like "all ProductVariants with the 'electronics' FacetValue whose name contains the word 'camera'."

Newly with this version, all the built-in filters now support a "combination mode", which allows you to specify the boolean operator used to combine multiple filters.

New built-in filters

We've shipped two brand-new CollectionFilters which allow you to directly select specific Products or ProductVariants to include in your Collection.

Improved list UX

Working with the Collection list just got a whole lot easier, as the state of the page (i.e. which collections are expanded, any active filter term) is now persisted to the url. This means you can filter the list, edit a Collection, use the back button and get the exact same filtered view again.

Other notable features

• Health checks: Some users reported that the database health check would give false negatives, i.e. the built-in timeout of 1000ms would sometime result in a report that the database was down when it wasn't. To solve this, we've re-architected the health check module and introduced a new HealthCheckStrategy to allow you to have full control over even the built-in health checks.

  import { TypeORMHealthCheckStrategy } from '@vendure/core'; export const config = { // ... systemOptions: { healthChecks: [ // increase the timeout from the default 1000ms // to avoid false negatives new TypeORMHealthCheckStrategy({ timeout: 5000 }), ], }, };

• Admin UI styling: We've made some small improvements to the visual styling of the Admin UI - most notably with a refreshed login screen and app header. If you'd like to remove the new "vendure" wordmark from both, you can always turn this off in your config.

• Unique custom fields: You can now specify that custom fields must be unique by setting unique: true, which will add a database UNIQUE constraint on the field.

• SearchEvent: A new SearchEvent has been introduced, which is published whenever someone performs a search query with the DefaultSearchPlugin or ElasticsearchPlugin. This can be used to implement search analytics for example.

• Payment & Shipping handlers: The configurable operations related to payments & shipping (e.g. PaymentMethodHandler and ShippingEligibilityChecker now receive the respective PaymentMethod or ShippingMethod to which they belong. This provides greater flexibility in supporting a wider variety of use-cases.

• OrderLine featuredAsset: If a purchased ProductVariant has a featuredAsset, it will be used in the OrderLine rather than the parent Product's featuredAsset.

• Braintree plugin: In the Braintree payment plugin, it is no longer necessary to pass the orderId when generating a payment token. Your existing code will not break, but you'll start to see a warning logged that this argument is deprecated. To fix the warning, just call generateBraintreeClientToken with no arguments from your storefront.

Thank you to all contributors

I'd like to thank the wonderful Vendure community who contribute their ideas, bug reports, and code to the project daily. This release includes code contributions by:

• Alexander Shitikov:

  • fix(core): Copy context on transaction start. Do not allow to run queries after transaction aborts. (#1481)

  • feat(core): Pass shipping method to calculator and eligibility checker (#1509)

  • fix(core): Manage transactions outside of orderService.modifyOrder function. (#1533)

  • fix(core): Job update doesn't emit if progress didn't change (#1550)

  • feat(core): Pass payment method to handler and eligibility checker (#1564)

• Daniel Stehlik:

  • fix(core): Truthy check for custom fields in importer

• Julien:

  • fix(payments-plugin): Send 200 response from Stripe webhook (#1487)

• Kevin Mattutat:

  • feat(event): Added event to track search inputs #1553

• Sebastian Geschke:

  • fix(payments-plugin): Fix state transitioning error case in Stripe webhook (#1485)

  • feat(core): Make search strategy configurable via plugin options (#1504)

• Toche Camille:

  • fix(core): Add missing OrderLine.order field resolver (#1478)

• waltheri:

  • fix(ui-devkit): Wrap output path in quotes. (#1519)