Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [rdf4j-dev] Multithreaded access to the MemoryStore
  • From: "Bart Hanssens (BOSA)" <bart.hanssens@xxxxxxxxxxxx>
  • Date: Tue, 9 Nov 2021 22:20:39 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=bosa.fgov.be; dmarc=pass action=none header.from=bosa.fgov.be; dkim=pass header.d=bosa.fgov.be; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xSB1RMppoZR+fLkfe1n+bz4ovuxboWu2llMzIIHd1EE=; b=EHeR4Spgy9big2YXz0nKQ5gitVJwXiLkt0nzV3K0TP2Bm3MFM6/RNojyXMb8f/Q3vlWuxep1qJVDkvtJlI/c693Q/SU4Gj4KdzTlllVXN4KbbCkRr+mA2vvLrBVMdRjjAiL8NwzGLIvZ2cIclnGe0JNivzcFAgqFXWuq7kcf4SPvtqRAHD/+rneU766TUBft6pP5gJ1houXuiCy+AeeyDlEOTqTCBF5Wz65Gi09d2VSvSMAOCIkoxNchHf3oQ0aMDUY9qPcPd5OI7w39z0BkSF+U1lGbjO2A14f8DdOKpsoolhiEkMvVK2N9uW0dh96xLWiQcVT22kSkkYS66M7Pyg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OrNWTxys/uziPPYxwMT1ga0rUxvETh+oVB3wlybOUzgXvHDIn6RUsQb7bxTXRxwCF29/uN2tih2nI8n67Ntd1/EtE9RiE0tFzMsKs+z0SB3/5kesI/UVwaq6DS/T+p0uYh/rdxzazEzb866Qyh9J3NmIVEYjEf6IH80cgZMQmHiCOEm2SmVd7BejH1bjqqvS8tQJoMWGvAjj8P/dhwNmbeEZGG8s0zrN2fqPhY+C5i9/u0KlgKgYJMT/KKI3ZpoyR3bmQIpBc1v+ZjOENDPWh8As5SNPKXV0DRLuSoi53c0gew+EOWrL3fy0OEFBQ6A2rv7a8bvPF5LHEaP8amO3pg==
  • Delivered-to: rdf4j-dev@xxxxxxxxxxx
  • List-archive: <https://www.eclipse.org/mailman/private/rdf4j-dev/>
  • List-help: <mailto:rdf4j-dev-request@eclipse.org?subject=help>
  • List-subscribe: <https://www.eclipse.org/mailman/listinfo/rdf4j-dev>, <mailto:rdf4j-dev-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://www.eclipse.org/mailman/options/rdf4j-dev>, <mailto:rdf4j-dev-request@eclipse.org?subject=unsubscribe>
  • Thread-index: AQHX1TL4gP6h1p8Zl06C8ZHjkMo44Kv7CnrAgAC55ZA=
  • Thread-topic: [rdf4j-dev] Multithreaded access to the MemoryStore

Hmz, I get quite a few exceptions (java.lang.IllegalArgumentException: Input stream must not be 'null') when running the set of SHACL benchmarks…

Is this expected behavior ? Or could this be because I’m running the tests from the command line (mvn -jar jmh-benchmarks.jar ?) and not inside a GUI ?

 

Best regards

 

Bart

 

From: Bart Hanssens (BOSA)
Sent: dinsdag 9 november 2021 12:49
To: rdf4j developer discussions <rdf4j-dev@xxxxxxxxxxx>
Subject: RE: [rdf4j-dev] Multithreaded access to the MemoryStore

 

Hi Håvard,

 

Haven’t used multithread access to the memorystore yet…

I do have access to a cheap “server” (consumer grade AMD Ryzen 5 3600 with 6c/12t) and JDK 17, so I’ll give that a spin.

 

Best regards

 

Bart

 

From: rdf4j-dev <rdf4j-dev-bounces@xxxxxxxxxxx> On Behalf Of Håvard Ottestad
Sent: dinsdag 9 november 2021 7:23
To: rdf4j developer discussions <rdf4j-dev@xxxxxxxxxxx>
Subject: [rdf4j-dev] Multithreaded access to the MemoryStore

 

Hi,

 

I’ve upgraded my laptop just a few days ago to a shiny new MacBook with 8+2 ARM cores. It’s a nice upgrade to my 9 year old MacBook with a quad core Intel processor. 

 

Things are definitely snappier, but I’m struggling to get much performance improvement when comparing single threaded SHACL validation to parallel validation. 

 

Profiler says that there are three major issues of overhead and contention are:

 - creating read locks

 - retrieving the mem-versions of values from the mem value store

 - keeping track of active iterators


Might be that the arm version of jdk 17 still needs optimizations to make locks faster, or maybe it’s due to the arm memory model being more relaxed than the Intel one. 

 

Anyone else run into the same issues?

 

Or someone with an 8+ core Intel/amd machine who would like to test it out? I’m using the benchmark linked below and not seeing much difference when enabling/disabling parallel validation. 

 

 

Cheers,

Håvard


Back to the top