eBay

Tracking events from over 162 million users means Big Data is eBay's biggest asset. But when that behavior and performance data is stored across multiple systems and using different technologies, synthesizing it all to make it useful to a team of 12,000+ analysts is a huge challenge on many levels. My role as the Director of Platform Experience was to make the concept of "community analytics" a reality and design a unified internal platform to support this bold new vision.

With over 52 petabytes and over 3.5 trillion rows of data, the size of Big Data at eBay can be mind-boggling. 100TB of new data is created every day. There are over 500 million live auctions in 50,000 product categories at any time and $3,500 of goods sold every second. That means small changes have a huge impact.

The vision was to create a single point-of-entry analytics portal from separate applications that empowered the business. Analysts can’t and shouldn’t work on complex data in a bubble. They need to collaborate to derive deep data insights. Taking a common collaborative concept and spirit from large open source projects, it was necessary to build a culture of collaboration and community analytics by providing a platform that would easily enable the sharing of large datasets and ideas. This also included a tool to unify a dozen other tools managing over 52 petabytes of data—therefore, the code name for the project became "Harmony".

The goal: create a single point-of-entry analytics portal that empowers the business.

We made the core element of the platform the "Insight". Insights are standalone concepts derived from data, often cross-referencing datasets to reveal aspects of the data not readily apparent. Once discovered, the Insight could be saved as its own discreet entity and subsequently shared, altered, or built-upon to the degree that it potentially evolved into a completely new Insight through a collaborative process. Insights could even be certified based on financial, legal compliance, or other credibility criteria making them more valuable. The power of this new app-like concept of Insights was virtually limitless.

The team was assembled from existing internal people, but it needed to be built out to support the scale and scope of the project. In the interim while I did that, I took on most of the team roles until we had a dedicated person in place. Because the different systems were owned and maintained by different teams, resources were also leveraged from those teams company-wide.

The scope of the project required performing the same research and design for each system to be integrated. The process of Task Analysis provided insight into how each system worked and helped determine where the different systems overlapped as well as the feature gaps. The systems were divided into four key areas based on primary function:

• Reporting: Piwik, MicroStrategy, Vulcan, dashboards, and spreadsheets
• Tools: Tableau (advanced visualizations), account provisioning, VDMs, Data Loader, Data Mover
• Data: Teradata (EDW, high-performance enterprise relational databases), Hadoop (distributed processing), Confluence (Wiki), proprietary metadata
• Social: Joomla, Kunena, mTree, and other components

The user personas consisted of technically sophisticated data scientists and moderately savvy sales directors. My process showed how different users used the same feature in different ways and helped reconcile anecdotal information with reality. For example, a data scientist will manually create a database query to get the data he needs and a sales director will prefer using the GUI to perform the same task. I also discovered a lot of duplicate effort during this research.

By collecting and aggregating user tasks, a Mental Model was created for each system and then superimposed onto a Global Mental Model to show how they all interoperate. The resulting Information Architecture had to be simple enough, yet go deep enough to handle the breadth of features. The primary scope of tasks centered around creating and customizing reports, so these were surfaced prominently in the menu options.

Users needed to start with data reports and visualizations to derive insights. To make these easier to generate, we created a centralized directory for pre-configured "widgets" where users could share work they'd already created. Widgets are app-like and could be reports, visualizations, or insights and were organized by a tagging system. Widgets could be searched, rated, annotated, copied, edited, or certified based on specific legal or financial criteria. Each widget the users created contained SQL queries and insights, which are an investment in the future of eBay's machine leaning techniques and increase the rate of forecast accuracy.

The primary UI was created around the concept of a WYSIWYG frame called a "workspace" where widgets could be added, configured, resized, and shared. Users could create as many workspaces as they desired and share them with team members. This was the starting point for users at the start of each day, automated and customizable with the information they needed. This helped reduce repetitive effort so analysts were not creating queries and reports which already existed.

Through our research, I discovered that nomenclature was a problem when referring to and using the various applications. Often, the same feature in three different applications had three different names. To create alignment on the naming used in Harmony, I aggregated all the language into a spreadsheet, including definitions and clarifications for each term. Then we tested those terms with users to narrow down the best overall consistent nomenclature.

To support the project vision of integrating a sense of community, we added an integrated discussion forum, employee profiles, and similar social features to provide a means to share and communicate.

We created two visual concepts based on the eBay branding guidelines:

• "Sunshine": A bright and open layout mainly using the eBay yellow and blue. A friendly and approachable design that reflected the new social aspects.
• "Atomic": A minimalist layout that used only shades of gray and the eBay blue as a highlight. A simplified design that got out of the user's way and allowed the analytics to be the "star".

The stakeholder consensus was to go with the "Sunshine" concept and keep the code name Harmony as the final name of the platform. However, the visual design process sparked deeper stakeholder discussions and it was decided to revisit the naming and UI.

So I launched a parallel process to determine the find product name and brand. I engaged all the stakeholders in a working session to define the parameters for the new name. Using a thesaurus, we created a list of attributes/adjectives that would best describe how they felt about each. I followed this up with a brainstorming session to clarify the goals of the new name, its relevancy and how it should resonate with the users. The consensus was to strip away everything down to its core and that "Harmony" was "too California" and "too touchy-feely". They preferred something descriptive as opposed to an experiential or invented name. In a subsequent whiteboard session, we created a matrix of key attributes and naming options, and then organized and aligned them to what attribute each name evoked. That process resulted in a simple, descriptive name for the platform: "Datahub".

Agile development was critical to the success of the project. We were able to shift gears in several cases to accommodate a change in requirements or business mandate. For example, once we built the first limited prototype ready for validation testing and changed the application name, the stakeholders and users decided they didn't like the Sunshine design concept in practice and overwhelmingly preferred the Atomic concept—“Let the data be the star.”

During our validation and refinement phase, we mostly hit the mark with the exception of a few key technical issues and optimizations. We also added a select menu option to the workspace tabs to solve a scalability issue and support virtually unlimited report views.

The most significant result was people were able to do their jobs better and more efficiently. That meant events from over 162 million eBay buyers could be captured, transformed, and synthesized to provide actionable insights for the business in the easiest and most engaging way possible.

An internal satisfaction survey conducted after the soft launch revealed a few minor improvements in caching and UI optimizations, but overall the response was very favorable. There was increased engagement and collaboration in data analysis, insights and forecasting as well as improved productivity and efficiency in on-boarding new analysts.

More information:

• eBay Selects Joomla! Open Source to Foster Innovation From the Inside Out
• eBay deploys Joomla for analytics portal
• eBay to use Joomla as the 'open source glue'