Making Production Hadoop Faster, Easier and More Productive: ODPi Member Conversation with Zettaset

By | Blog

In our fifth ODPi Member Conversation podcast, John Mertic spoke with Zettaset’s Director of Product Management, Sesh Ramaswami, and CMO, John Armstrong.

Their multi-perspective discourse covered off on the security challenges of deploying Hadoop in production, data governance and security expectations, along with how nonprofit organizations like OASIS and ODPi can help drive standards for the big data ecosystem.

Touching on big data security within production environments, Ramaswami dove right into enterprise expectations around data governance and data processing.

Referring to the role data governance plays in the Hadoop ecosystem, Armstrong noted that security wasn’t always a concern, as data was in isolation away from production.

However, the space has changed drastically over the years. Touching on this shift, Armstrong said, “When [enterprises] finally want to move into production, and want to take those data environments and put them into the mainstream… we need to have some real security in place before you plug into the rest of the network —  because Hadoop coexists in large enterprise environments in conjunction with relational database environments, maybe other NoSQL, so it’s not alone and it has to play well with others.”

To hear more of the group’s insight, tune in to the episode on our YouTube channel!

Subscribe to our YouTube channel and follow us on Twitter to catch upcoming episodes of the ODPi Member Conversation podcast series!

Making Production Hadoop Faster, Easier and More Productive: ODPi Member Conversation with IBM

By | Blog

In our fourth ODPi Member Conversation podcast, John Mertic met with IBM’s Worldwide Analytics Architect Leader for Data Lakes, Neil Stokes.

Their insight-rich conversation kicked off with a discussion around machine learning and cognitive vision, along with how an ODPi Compliant Hadoop environment relates to cognitive computing.

Stokes, who has spent more than 20 years working at IBM, has a unique perspective on how Hadoop and cognitive systems afford enterprises the ability to ingest and make sense of the rawest possible forms of data, and why systems are only as good as the corpus of data to which it has access.

Referring to these sets of broad, invaluable data, Stokes noted “Interoperability is key here… Groups like ODPi – who have the ability to facilitate that level of interoperability across vendors, across platforms, across different technologies – there is a tremendous value that groups like ODPi bring [to the ecosystem].”

To hear more of Stokes and John’s discussion, tune into the episode on our YouTube channel!

Subscribe to our YouTube channel and follow us on Twitter to catch upcoming episodes of the ODPi Member Conversation podcast series!

Twitter Poll Results: Is Apache Hadoop Running in Production?

By | Blog

Following the publication of our White Paper, 2017 Preview: The Year of Enterprise-wide Production Hadoop, we ran a series of Twitter polls to get a rough sense of where the market is on the following 4 questions:

  1. Does your company use Hadoop in production?
  2. What stage are most of your Hadoop deployments (lab, PoC, Pilot, Enterprise-wide production)
  3. When will you have Hadoop in Enterprise-wide produciton use?
  4. What challenges did you encounter while expanding Hadoop use?

We started with the basics, asking first:

The split between production and non-production use is in line with what we hear from our community.

As we discuss at length in the white paper, this concept of “production” Hadoop can be misleading. For instance, pilot deployments and enterprise-wide deployments are both considered “production,” but they are vastly different in terms of DataOps, as table 1 below illustrates.

Table 1: DataOps Considerations from Lab to Enterprise-wide Production

In the next poll, we learned that 72% of Hadoop deployments are stacked up in the pre-enterprise wide stages.

One of the other diagrams you’ll find in our white paper is the Enterprise Hadoop Deployment Continuum. In the version below, I have added the percentages from the Twitter poll in each stage.

Figure 1: Most Hadoop deployments are in pre and limited production.

With this established, we then asked the Twitterverse when they expect to be enterprise-wide with Hadoop? Reassuringly, the same 28% that told us they were enterprise-wide in poll #2 reiterated this in poll #3.

Less reassuring, however, is that only 9% of those that are presently pre-enterprise wide have concrete plans to get into enterprise-wide in the next 12 months, and even fewer have such plans in the next 24 months.

An eyebrow-raising 55% said that they’re not sure when they will reach enterprise-wide deployment.

And when asked about the challenges big data pros faced increasing their use of Hadoop, responses were very evenly distributed across the four big areas we hear from the ODPi community.

ODPi is here to remove risk and uncertainty from Hadoop and Big Data. We do this through comprehensive testing suites that improve predictability and through compliance programs to ensure interoperability. In other words, ODPi is here to smooth and illuminate the path to enterprise-wide production use of Hadoop for the 55% of respondents that don’t know when (if?) they will get there.

And the ODPi Special Interest Groups, or SIGs, were set up to address the widespread challenges that poll #4 surfaced.

Like all the technical work at ODPi, SIGs are wide open for all to participate in.

Join us and help drive toward solutions in these areas.

Making Production Hadoop Faster, Easier and More Productive: ODPi Member Conversation with zData

By | Blog

In our third ODPi Member Conversation podcast, John Mertic met with zData’s Senior Solution Architect, Gagan Brahmi.

Their in-depth conversation centered around the struggles commercial and enterprise corporations face with Apache Hadoop deployments, along with why standardization is an important driver in the Big Data community.

Brahmi has a unique perspective on Hadoop’s role in the Big Data industry and offered specific insight around the ways today’s enterprises can derive the maximum value out of the Hadoop cluster and their data.

Touching on how the framework’s capabilities have grown tremendously in the last few years, and urging enterprises to recognize how many tools throughout the ecosystem now complement one another, he noted “When we are dealing with [this] tool to make sure we derive the maximum value out of our existing systems, we have to keep an eye on all the tools available.”

To hear more of Gagan and John’s discussion, tune in to the episode on our YouTube channel!

Subscribe to our YouTube channel and follow us on Twitter to catch upcoming episodes of the ODPi Member Conversation podcast series!

Making Production Hadoop Faster, Easier and More Productive: ODPi Member Conversation with Pivotal

By | Blog

In our second ODPi Member Conversation podcast, John Mertic sat down with Pivotal Software’s Head of Data, Jacque Istok.

Their engaging discussion focused on the challenges enterprises face when trying to get value out of their data in Hadoop.

As a founding member of ODPi, and former technologist with a long history in data warehousing and data analytics, Jacque also weighed in on standards and how they enable application interoperability, portability, governance and security across the ecosystem.

Tying the importance of standards into his thoughts around ODPi, he noted “Our vision has always been to make it easier for customers and products/vendors/projects to be able to interact with an enterprise standard for Hadoop in an easy and common way.”

To hear more of Jacque and John’s insight, including why it was important to Pivotal that ODPi was a core organization with a common goal for the Hadoop platform itself, tune in to the episode on our YouTube channel!

Subscribe to our YouTube channel and follow us on Twitter to catch upcoming episodes of the ODPi Member Conversation podcast series!

Stay Informed

Sign up for our Newsletter to receive the latest ODPi news and updates.