Building an Integrated Platform for BI and Community Analytics
The Challenge
eBay’s massive data operation tracks over 162 million users, but valuable information was fragmented across disparate systems using incompatible technologies.
In my role, my goal was to transform isolated data silos into a unified “community analytics” platform, serving over 12,000 analysts and revolutionizing how eBay leveraged its most valuable asset: data.
The Strategy
The scale was immense: 52 petabytes of data across 3.5 trillion rows, with 100TB generated daily from 500 million auctions in 50,000 categories. In this environment, both minor errors and improvements had major business impact.
Our strategy focused on creating a collaborative analytics ecosystem, not just another siloed tool. Through extensive market analysis and internal research, we recognized the need for enhanced social collaboration to leverage eBay’s collective intelligence. I made a strategic decision early on to integrate social collaboration features, allowing users to share and build on insights seamlessly.
I established the foundational concept of the "Insight" — a standalone, shareable data interpretation that could be built upon collaboratively (similar to the “app directory” concept). My strategy ensured these Insights could be certified for compliance, shared across teams, and evolved through collective refinement, creating an ever-improving knowledge base for the entire organization.
The Process
I assembled and led a cross-functional team, initially wearing multiple hats while building out specialized roles. I directed a systematic research approach across each system to be integrated, conducting qualitative research with teams to understand how they used various tools, their workflows, and future needs. Under my leadership, we:
- Conducted deep task analysis that revealed how each system functioned and where they overlapped or left gaps
- Led user research across varied personas (from data scientists to sales directors) to uncover different interaction patterns with the same features
- Created mental models mapping how each system's concepts related to build a coherent global information architecture
We organized the underlying systems into four functional categories:
- Reporting: Piwik, MicroStrategy, Vulcan, dashboards, spreadsheets
- Tools: Tableau visualizations, account provisioning, data loaders/movers
- Data: Teradata enterprise databases, Hadoop distributed processing, metadata systems
- Social: Community platforms for knowledge sharing and collaboration
Based on user research insights, I drove the interface design around supporting two key concepts:
- Widgets: Reusable, shareable components containing reports, visualizations, or insights
- Workspaces: Customizable dashboards where widgets could be arranged, configured, and shared
I addressed terminology inconsistencies by creating a unified glossary, testing language with users to ensure clarity across previously disconnected systems.
Two competing visual design directions emerged:
- "Sunshine": Bright, open layouts using eBay's yellow and blue brand colors, emphasizing collaboration
- "Atomic": Minimalist interface using gray with blue accents, letting the data visualization take center stage
I facilitated iterative naming workshops and attribute mapping sessions, transitioning from the code name "Harmony" to "DataHub"—reflecting the platform's role as the central nervous system for eBay's analytics.
Adopting an Agile methodology allowed our team to pivot quickly. For example, when usability tests indicated strong preference for a minimalist (“Atomic”) interface—“Let the data be the star”—I decided we should pivot away from the original “Sunshine” design direction.
The Results
The DataHub platform I designed and delivered transformed how eBay's 12,000+ analysts worked with data, enabling them to:
- Discover and leverage existing analyses rather than recreating them, thereby saving countless hours
- Collaborate across previously isolated teams to derive deeper insights
- Accelerate analyst onboarding through standardized tools and unified terminology
- Produce reliable, certified insights driving critical business decisions
Post-launch surveys indicated overwhelmingly positive feedback, revealing improvements in forecasting accuracy, efficiency gains, and significantly increased cross-team collaboration. Under my leadership, the platform significantly increased collaboration in data analysis, improved forecasting accuracy, and reduced redundant work—ensuring that the massive data from 162 million eBay buyers was transformed into actionable business intelligence more efficiently than ever before.