Datameer 1.4
Here are the new features and significant changes implemented in Datameer 1.4
Partitioning Data Sets
Users can now create time-based partitions from data stored by Datameer in HDFS (Run Import Job, File Upload or Workbooks). When working with data (Link Data Set, Link External Sheet, Create Export or View Full Data), users can now select subsets of the data by either static (from-to date) or dynamic (e.g. last week) date ranges increasing both performance and usability. Partitions can be reconfigured as needed with a few clicks.
Multi-way Join
When joining a sheet with many lookup tables, this can now be accomplished within a single dialog without the need for additional sheets.
New Field Types
Datameer now supports big integer and big decimal data field types. These allow the user to work with very large numbers as well as with a very high precision. In addition, the Datameer plug-in SDK allows users to configure custom scale and precision values for big decimals when writing their own input adapter.
Separate Files for Field Detection
In cases where the first rows of a data file do not contain data representative of the entire data set, users can upload and specify a separate header file, which is then used for field and type detection.
Input Adapters
Datameer now has connectors for data in both PostgreSQL and Greenplum databases.
Enhanced REST API
Several enhancements have been made to the Datameer REST API making it more flexible when working with 3rd party applications or scripts.
Security Enhancements
Secure LDAP
Datameer has added support for Secure LDAP (over SSL) as an external authentication mechanism.
Group Filters
Group filters can be applied when importing users from Active Directory or LDAP.
Secure Hadoop Sychonization
In addition, with secure impersonation enabled, Datameer leverages existing HDFS permissions and enforces these at the end-user level. Datameer also secures data in its own private folder based on the (authenticated) user. In this way, permissions to all new or pre-existing user-specific data are now completely synchronized between Datameer and Hadoop.
More Supported Platforms / Hadoop Distributions
Datameer now supports Cloudera’s Hadoop CDH3u1 and CDH3u2, IBM’s BigInsights 1.2 (IHC) including GPFS.
See Supported Hadoop Distributions for details.
New functions
MINUTE - retruns only the minute of a date.
GROUPANY - returns an arbitrarily selected value contained in the group.
GROUPFIRST - returns the first value contained within the group.
GROUPLAST - returns the last value contained within the group.
API Changes
We have made a few important changes to our API. Please see Important API Changes for Developers for more information.
Oracle CLOB Support for Importjobs
A character large object or CLOB usually have very high size limits on the order of 2GB or more. Since Version 1.4.4 we support the import of clob columns as strings.