Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [rdf4j-dev] NativeStore and file sync

It's disabled by default because, if I remember correctly, its performance is very poor on larger update operations. However, you are correct of course that it somewhat compromises on the failsafe aspect of the store. 

I'm not against making it enabled by default, especially if we can combine that with certain options to cater for bulk uploads or other things to mitigate performance loss. I would like some recent comparative benchmarks though to see what the actual performance degradation is on various kinds of update operation. I can live with some degradation, but if we're suddenly 150% slower on every write operation, that's a bit much.

As for bulk uploads, not sure what you have in mind there but one relatively simple option might be to look at the transaction isolation level. For example, we could enable force sync by default, but then turn it off if a transaction with isolation level NONE comes in (which is typically the isolation we recommend using for bulk uploads). This is an off the cuff idea and I have no idea how that would play out in the face of concurrent transactions, to be honest, but it might be worth looking at.

Cheers,

Jeen

On Tue, Apr 21, 2020 at 1:13 AM Håvard Ottestad <hmottestad@xxxxxxxxx> wrote:
Hi,

While looking at the NativeStore I've noticed that there is a feature to enable forced sync for file writes. From my understanding this would be used to force the underlying storage to flush writes to disk in the case that it is still only in a write cache in memory.

This guarantee is particularly useful for a database, since people tend to trust them more in the face of failures (power failures in this case).

Forced sync is currently disabled by default.

I'm thinking we should enable it by default. Even though this will degrade performance.

I would also like to introduce a bulk loading method that would use the optimal settings for loading data (including turning off forced sync).

What do people think about this?

Håvard
_______________________________________________
rdf4j-dev mailing list
rdf4j-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/rdf4j-dev

Back to the top