Re: [geomesa-users] Compilaiton error in Java when applying "map" to an

[Luca Morandini <luca.morandini1@xxxxxxxxx>]

On 09/05/15 07:30, Jim Hughes wrote:
> Hi Luca,
>
> Great question.  Sadly, I haven't used Spark from Java, nor have I used Java 8.
...
> Defining the scala.Function1 directly should work though:

Indeed it did :) Thanks for taking the time to help me, navigating between Java 
and Scala can be rather difficult.

However, I tripped on another Java/Scala issue when running an integration test :(
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 24.513 sec <<< 
FAILURE!
testTransaction(au.org.aurin.tweetcommons.JavaRDDTweetIT)  Time elapsed: 16.734 
sec  <<< ERROR!
java.lang.IllegalArgumentException: null
        at 
com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
        at 
com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
        at 
com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
        at 
org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$$getClassReader(ClosureCleaner.scala:41)
        at 
org.apache.spark.util.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:84)
        at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:107)
        at org.apache.spark.SparkContext.clean(SparkContext.scala:1622)
        at org.apache.spark.rdd.RDD.map(RDD.scala:286)

I googled around and found out it is an issue with a library used by Spark:
https://issues.apache.org/jira/browse/SPARK-6152
https://github.com/EsotericSoftware/reflectasm/issues/35

I am currently using 1.0.0-rc.5 and Spark 1.3.0, shall I start using newer versions ?

Regards,

Luca Morandini
Data Architect - AURIN project
Melbourne eResearch Group
Department of Computing and Information Systems
University of Melbourne
Tel. +61 03 903 58 380
Skype: lmorandini