Here is a summary of what was achieved.
Redundant calls for OPTIONS #287
Work: Added GET method as replacement of POST method for /api/-/query endpoint.
Results: The added GET method is a CORS simple request which doesn’t trigger an OPTIONS request. Additionally the GET method can be
cached by a load balancer.
Add Spring caching #280
Work: HTTP responses are already cached by a load balancer and Hibernate caches managed database entities. Instead of adding an additional
caching layer, the decision was made to optimize the /vscode/gallery/extensionquery endpoint to research how performance can be improved.
Results: The amount of database queries is greatly reduced by fetching results upfront in bulk instead of one by one while composing
the JSON response. Data transfer objects (DTOs) were used instead of managed entities to minimize Hibernate overhead. The downside of this approach is that the resulting queries are verbose string literals that might hurt the project’s maintainability in the
future.
Add more cache directives to the server #266
Work: Added Cache-Control response headers where possible.
Results: Responses are cached correctly by the load balancer.
Download Count seems off #283
Work: Configured Azure Blob Storage metrics. Added download count service that pulls in the metrics, extracts the downloads and updates
extension download counts.
Results: Open VSX server can keep caching download redirect responses, while accurate download counts are pulled in from Azure Blob
Storage. After being broken for an extended period, download count is now working again.
Search leads to unexpected results #244
Work: Added search by extension id (namespace.name) as a first step. No further search algorithm improvements were needed.
Results: Search no longer leads to unexpected results. Users can now search by extension id.
[Admin] Server Error Removing Extension #285,
was #447
Work: Removing an extension failed, because there was a mismatch between the expected and actual query parameter type. The issue was
resolved by changing the database queries that search for dependencies and bundled extensions.
Results: Admin can remove extensions again.
Deployment Statistics in the Admin Dashboard #235
Work: Added /admin/report endpoint that provides monthly statistics in CSV format.
Results: Admin can request monthly statistics through the admin REST API.
Upload fails with an Internal Server Error #343
Work: Increased extension version tags column size. Added deduplication and sorting logic for new and existing extension version tags.
Results: Tags are deduplicated and sorted the same way as the Visual Studio Marketplace.
Optimize performance for /api/-/query endpoints #356
Work: Reduce database queries by fetching results upfront in bulk. Minimize Hibernate overhead by using DTOs.
Results: After deployment of the changes the open-vsx.org production
server saw a 20x decrease in the amount of database transactions needed (from 10.000 to 500 transactions) and a 5x decrease in peak CPU usage (from peaks hitting 5 to peaks slightly over 1).
Use jOOQ instead of DTOs to improve maintainability #358
Work: Configure jOOQ. Rewrite DTO queries to use jOOQ instead.
Results: The rewritten queries will break when the database schema changes, so that errors are discovered in development and can be
quickly fixed. String literals are more error prone in general (typos, missing space or comma) than object oriented queries. Using jOOQ improves the maintainability of the Open VSX project.
Overall, the Open VSX project is in a better position for further growth and adoption.
The server performs better, and by refactoring other REST endpoints in a similar way the server can be improved further. Critical usability issues regarding publishing, deleting and searching
extensions were resolved. Extension download counts are updated again, after being broken for an extended period.
Special thanks go out to:
Miro Spönemann, for introducing jOOQ to the project and reviewing my pull requests.
Anton Kosyakov, for testing the optimized REST endpoints and reviewing my pull requests.
Mikael Barbero and Frederic Gurr, for helping test the server in the staging environment.
Brian King, for his understanding, guidance and making this sponsorship happen.
