Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] plan for supporting newer accumulo releases?
Hi Simon,

Check out the new branch named rc7_a1.7_h2.5.(1)  I managed to get it build cleanly, but due to limited time, I had to skip fixing some small api changes with the jobs and tools sub-modules.

Those sub-modules should not be crucial to general use, and this should give you a chance to try out Kerberos with GeoMesa.  In general, if you have questions/issues with how GeoMesa interacts with Accumulo's security features, we'd love to hear about it and help address issues.

I'll definitely check out how Spark's build works.  The same trick with -Dhadoop.version should work for GeoMesa's build (since one is just overriding a Maven property).  The biggest challenge is dealing with API versions.  Using an override to bump between Accumulo 1.5 and 1.6 or 1.6 and 1.7 would result in build errors.  The same is likely turn for Spark, and hence they need to use the profiles to help address those concerns.  That said, the -D trick would be great for localizing to a Cloudera or Hortonworks version of dependencies like Hadoop/Accumulo/etc.

Thanks,

Jim

(1) https://github.com/locationtech/geomesa/tree/rc7_a1.7_h2.5

On 05/29/2015 08:03 PM, Xu (Simon) Chen wrote:
Thanks Jim!

I am very interested in leveraging the latest kerberos auth support of accumulo 1.7:
Right now, we have to pass in accumulo user/pass to run a geomesa program, which is not very desirable.

I've tried accumulo 1.6 and that didn't work with geomesa master branch, clearly due to dependency issues. I haven't tried that branch yet, but directly to 1.7 is certainly better :-)

I am running CDH stack for hadoop/yarn, currently on 5.3.x release, which comes with zookeeper 3.4.5 and hadoop 2.5.0. CDH 5.4.x is also available, and it's based on hadoop 2.6.

In general, I like how spark handles dependency - you can package almost any version of hadoop client. For example, I built my spark setup with commands like:
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.5.0-cdh5.3.3 -DskipTests clean package

I'd be interested to know how you guys decide to go next and potentially help with some testing. But so far, our geomesa setup looks promising. Great job!

-Simon




On Fri, May 29, 2015 at 3:04 PM, Jim Hughes <jnh5y@xxxxxxxx> wrote:
Hi Simon,

Great question.  At the moment, we've been focused on Accumulo 1.5, and provided some support for 1.6 as the need has come up. 

There are some branches which support Accumulo 1.6 up on GitHub.  Of note, a1.6_h.2.6 handles some of the changes which are needed for Accumulo 1.6 and Hadoop 2.5/2.6. 

GeoMesa is at the exciting corner of several dependencies, so managing a complete set of everything one is a tad tricky.  I do want to provide better support as we continue.  In general, I'm happy to build a branch for a requested set of versions (Accumulo, Hadoop, Zookeeper).

Do you have a present need for GeoMesa to be capability with Accumulo 1.6 or 1.7?  If so, could you suggest some versions of Hadoop and Zookeeper which may be associated?  If so, I'm happy to build up a branch, and shake down any api/unit tests changes.  In exchange, I'd ask for some help identifying any gotchas while deploying iterators or the GeoServer plugin.

Cheers,

Jim


On 05/29/2015 02:44 PM, Xu (Simon) Chen wrote:
Hey folks,

I believe only accumulo 1.5 works with geomesa, as I previously had no luck with accumulo 1.6. And now accumulo 1.7 is released, which supports client kerberos auth - that's a valuable enterprise feature...

I am wondering if there is a plan to move geomesa to newer accumulo releases?

Thanks.
-Simon


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users



_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
http://www.locationtech.org/mailman/listinfo/geomesa-users

Back to the top