Elasticsearch for Apache Hadoop 5.0.0-alpha5


See issues on GitHub

Release Notes

New features

  • New Slice API:
    • Add the ability to create IndexPartition based on the desired number of documents per split #812 (issue: #778)
  • Spark:
    • Support for Spark 2.0 / Datasets #647

Enhancements

  • Rest:
    • Validate field exclusion settings when EsSpark.saveJsonToEs() is executed #782
    • Review the error messages in ES 5.x and properly parse them #779

Bug fixes

  • Rest:
    • Fixed parsing of ES mapping with OBJECT field named 'properties' #810 (issue: #809)
    • Not all errors in ES are reported when writing #720
  • MR and Spark:
    • Restore data locality preference for hadoop and spark #819 (issue: #814)
  • Spark:
    • saveToEs saves fields with NULL values #792
  • Yarn:
    • Authenticate with a local keytab in the YARN Application Master #807

Non-issue

  • Spark:
    • Got exception when I tried to invoke _mtermvector #796
    • ES-Hadoop on Spark 2.0 #759

Documentation

  • HDFS Repository:
    • Repository HDFS doc improvements #781
    • Snapshot Creation Exception #813
  • Rest:
    • Corrected property name #805

Known Issues

  • Core ES-Hadoop:
    • Restore data locality preference for hadoop and spark #814
    • Update/Upsert bulk actions with scripts are broken for ES 1.x as of 5.0.0-alpha5 #817