Sorry for the delay getting back to you. We tried 4.5.4 and it works pretty well. I found one NFS-related bug and uploaded a fix [1]. We're also seeing MissingObjectExceptions that we think are NFS related, but we're not certain. The setup we have for this instance is a master replicating over git:// to git-daemon through a load-balancer to two slaves sharing an NFS filesystem. The slaves are running a custom git.git (has a few NFS error-avoidance patches) for repacking.
[2018-04-06 10:42:38,524] [SSH git-upload-pack 'XXXXXXX' (YYYYYYY)] ERROR com.google.gerrit.sshd.BaseCommand : Internal server error (user YYYYYY account ZZZZZZ) during git-upload-pack 'XXXXXXX'
org.eclipse.jgit.transport.UploadPackInternalServerErrorException
at org.eclipse.jgit.transport.UploadPack.service(UploadPack.java:768)
at org.eclipse.jgit.transport.UploadPack.upload(UploadPack.java:667)
at com.google.gerrit.sshd.commands.Upload.runImpl(Upload.java:80)
at com.google.gerrit.sshd.AbstractGitCommand.service(AbstractGitCommand.java:101)
at com.google.gerrit.sshd.AbstractGitCommand.access$000(AbstractGitCommand.java:32)
at com.google.gerrit.sshd.AbstractGitCommand$1.run(AbstractGitCommand.java:70)
at com.google.gerrit.sshd.BaseCommand$TaskThunk.run(BaseCommand.java:437)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:376)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.eclipse.jgit.transport.WantNotValidException: want 56c76057710dd3ca29e66ef6cdc4b727be781384 not valid
at org.eclipse.jgit.transport.UploadPack.parseWants(UploadPack.java:1197)
at org.eclipse.jgit.transport.UploadPack.processHaveLines(UploadPack.java:1072)
at org.eclipse.jgit.transport.UploadPack.negotiate(UploadPack.java:1052)
at org.eclipse.jgit.transport.UploadPack.service(UploadPack.java:745)
... 14 more
Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown 56c76057710dd3ca29e66ef6cdc4b727be781384
at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:158)
at org.eclipse.jgit.lib.ObjectReader$1.open(ObjectReader.java:300)
at org.eclipse.jgit.revwalk.RevWalk$2.next(RevWalk.java:971)
at org.eclipse.jgit.transport.UploadPack.parseWants(UploadPack.java:1184)
... 17 more
We're assuming this error happens due to NFS caching directory entries. We're looking at testing that out by changing the mount options (lookupcache, noac, ac*time), but these are prod hosts, so we don't want to hurt the performance too much. Do you have any ideas on how we'd approach solving this in JGit or a different hypothesis on what could be causing this error?