[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [rdf4j-dev] ShaclSail performance
|
Ok so when you said you were working on performance optimizations, I didn't expect you to open up another 11 concurrent pull requests immediately afterwards :)
As I said I'm fine with doing performance fixes in 3.7.2, but please do keep in mind that a patch release is supposed to minimize the risk of introducing new bugs. Doing a lot of performance fixes in the same part of the code (even if each individual fix is small) compounds that risk. Not too worried about it at the moment as you've done an excellent job with test coverage on the ShaclSail, but just something to keep in the back of our mind.
Jeen
On Sun, 25 Jul 2021, at 21:08, Jeen Broekstra wrote:
On Sun, 25 Jul 2021, at 19:19, Håvard Ottestad wrote:
Hi,
I’m working on performance optimizations for the ShaclSail so that bulk validation of large datasets works as expected. At the moment the bulk validation ends up keeping far too much data in memory and a few of the validation plans are too slow for bulk validation.
I would like to aim most of these performance fixes for a new bug fix release (3.7.2) since there are no new features or user facing changes. ,
Sounds good. With performance fixes, it kind of depends on the impact of the performance fix a little (in terms of amount of refactoring), but in principle I think it's fine to do that kind of thing in a patch release.
For 4.0.0 I am still aiming for even better bulk validation support. The fixes I have at the moment are just stop-gap measures to make bulk validation work again after I broke it with my big ShaclSail rewrite earlier this year.
4.0.0 will include a more thorough bulk validation support throughout the ShaclSail as well as a “large datasets detection” feature that automatically switches to bulk validation.
Sounds exciting!
There's a quite a few bug fixes already lined up for a 3.7.2 release, so we can release it as soon as you're ready I guess.
Cheers,
Jeen
_______________________________________________
rdf4j-dev mailing list
