Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] GeoMesa Docker EMR - Jupyter Notebook help
Hi Byron,

As happenstance, I'm setting up a GeoMesa demo, and I have a quick fix. 

You'll want to connect to the Jupyter docker (say, with 'docker exec -it jupyter /bin/sh), and edit this file: /var/lib/hadoop-hdfs/.local/share/jupyter/kernels/geomesa_spark_scala/kernel.json.

The line with the with the Toree Spark opts should read...

    "__TOREE_SPARK_OPTS__": "--driver-java-options=-Xmx4096M --driver-java-options=-Dlog4j.logLevel=info --master yarn --jars file:///opt/geomesa/dist/spark/geomesa-accumulo-spark-runtime_2.11-1.3.0.jar,file:///opt/geomesa/dist/spark/geomesa-spark-converter_2.11-1.3.0.jar,file:///opt/geomesa/dist/spark/geomesa-spark-geotools_2.11-1.3.0.jar",

One of the jars changed names (from geomesa-accumulo-spark_2.11-1.3.0-shaded.jar to geomesa-accumulo-spark-runtime_2.11-1.3.0.jar).  That difference caused the issues; I need to sort out re-building the Docker images.

Let me know if that doesn't sort it out!

Cheers,

Jim

On 02/14/2017 04:07 PM, Byron Chigoy wrote:

Hi - probably pretty basic, but we are able to get the Docker Bootstrap tutorial working on AWS. We are pulling from https://quay.io/organization/geomesa . Once started we can ingest the GDELT example and get the descriptive.  We are also able to bring the GDELT example into GeoServer. 


However while Jupyter gets docked - the Kernel GeoMesa Spark - Scala fails (Just says kernel busy).  We started the notebook on another port to see the error behavior and get a list of them. See below any help or clues would be most appreciated.


(Scala,org.apache.toree.kernel.interpreter.scala.ScalaInterpreter@5ef0d29e)
(PySpark,org.apache.toree.kernel.interpreter.pyspark.PySparkInterpreter@38f57b3d)
(SparkR,org.apache.toree.kernel.interpreter.sparkr.SparkRInterpreter@51850751)
(SQL,org.apache.toree.kernel.interpreter.sql.SqlInterpreter@3ce3db41)
17/02/14 20:50:29 WARN toree.Main$$anon$1: No external magics provided to PluginManager!
17/02/14 20:50:32 WARN layer.StandardComponentInitialization$$anon$1: Locked to Scala interpreter with SparkIMain until decoupled!
17/02/14 20:50:32 WARN layer.StandardComponentInitialization$$anon$1: Unable to control initialization of REPL class server!
17/02/14 20:50:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[W 20:50:34.769 NotebookApp] Timeout waiting for kernel_info reply from fe9c2776-f5d7-47bc-b5dd-d2769f631f2f
17/02/14 20:50:35 WARN yarn.Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
17/02/14 20:50:42 ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.

Byron




_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users


Back to the top