Executive summary
As machine learning and on-demand data processing became more widespread, YellowDog saw an opportunity to address a key computing challenge: how to give organisations greater flexibility over where demanding processing work was carried out.
Its vision was to bring cloud and in-house computing resources together within one platform, without requiring customers to manage the complexity behind each environment.
Working closely with YellowDog, Helmes designed and developed a hybrid, multi-cloud solution capable of distributing thousands of jobs across available resources. Customers could monitor progress in real time, while predictive capabilities helped estimate completion times and identify the likelihood of failures.
The platform went on to serve more than 2,500 users in its first year, with the technology used in sectors with demanding processing requirements, including financial services and animation.
Meet the customer
YellowDog is a Bristol-based UK technology company specialising in software for organisations that need to process complex workloads quickly and efficiently.
Its technology is used in areas where processing time can directly affect business decisions, from financial risk modelling to artificial intelligence. YellowDog’s focus is on removing computing bottlenecks, helping organisations make better use of available resources while balancing performance and cost.
The challenge
The processing power required for large-scale workloads can change quickly. YellowDog needed a platform that could respond to those changing requirements, drawing on capacity across different environments as demand changed.
For customers, that complexity needed to remain manageable. Users had to be able to:
- Monitor what was happening
- Understand how resources were performing
- Track the progress of their jobs without having to manage each underlying environment separately.
The platform also needed to fit into the way customers already worked. That meant supporting different computing environments as well as specialist industry software, simplifying existing workflows rather than asking organisations to reshape them around the platform.
Our approach
The engagement began by assembling a dedicated Helmes team around YellowDog’s product vision. Design, engineering, QA, DevOps and project management were brought together to support the product from early decisions through development and delivery.
Rather than treating design, development and delivery as separate stages, the teams worked across the product in parallel. This was particularly important for a platform where decisions about the underlying technology could directly affect what customers needed to see and control.
The collaboration brought together several areas of expertise:
- Shaping the product: Design sessions helped the teams work through requirements and translate YellowDog’s vision into practical product decisions.
- Making complexity understandable: Information architecture research helped determine how large amounts of operational and performance information should be organised and presented to users.
- Testing as the product evolved: QA formed part of the development process, helping identify issues as new capabilities were introduced.
- Supporting continuous delivery: DevOps and CI/CD practices supported the development, testing and release of changes throughout the engagement.
Close collaboration remained central throughout. YellowDog brought the product vision and understanding of its market, while Helmes contributed the design and engineering capabilities needed to translate that vision into a working product.
The solution
The resulting platform combined workload management, predictive capabilities, and performance monitoring within one product, supported by interfaces designed to make technically complex information easier to work with.
- Distributing work at scale
At the core of the platform was the ability to distribute large numbers of jobs across available computing resources. YellowDog’s rendering application, also developed by Helmes, processed jobs on those resources while providing customers with real-time information on their progress.
The platform could draw on both on-premises infrastructure and cloud capacity as needed, with support for providers including AWS, Google Cloud, Microsoft Azure and Oracle Cloud.
- Predicting completion and potential failures
Helmes also developed a prediction model using machine learning together with YellowDog’s proprietary algorithms. It estimated how long jobs were likely to take and the likelihood of failure.
For customers, this added greater predictability to demanding processing work, providing information that could support planning around job completion and potential failures.
- Making performance visible
A performance and analytics dashboard gave administrators, managers and analysts a clearer view of how jobs and computing resources were performing.
Users could review information across custom time periods, monitor resource use and export data for reports and presentations. Custom navigation helped users find key information within the dashboard containing large amounts of operational and performance data.
The interface was also designed for use across desktop and mobile devices.
- Connecting with existing tools
Helmes developed plugins that allowed YellowDog to work with specialist software already used by its customers, particularly 3D modelling tools and widely used rendering engines.
This helped the platform fit into existing workflows rather than requiring customers to move away from the software they already relied on.
Results & benefits
YellowDog gained early traction after the platform’s development, reaching more than 2,500 users in its first year.
Its customer base included animation studios Blue Zoo and Brown Bag Films, whose work has received BAFTA and Emmy awards. Content rendered using YellowDog’s technology went on to be distributed or broadcast through organisations including the BBC, Sky, ITV, Disney, Nickelodeon, Netflix and YouTube Originals.
YellowDog also received recognition from Oracle, including Disruptive Technology and Excellence in Innovation awards.
At particularly high levels of demand, the platform demonstrated the scale of computing resources it could coordinate. One production used enough GPUs that its processing performance, measured in petaflops, would have ranked alongside the eighth-fastest supercomputer in the world at the time.
Oracle Excellence in Innovation awards.
Photo by YellowDog.
Technologies
Languages:
- Java
- TypeScript
- JavaScript
- Python
- C#
Frameworks & platforms:
- Angular
- Spring
- MySQL
Cloud & infrastructure:
- AWS
- Oracle Cloud Infrastructure (OCI)
- Rackspace
- OVH
- Terraform
- Ansible
Integrations:
- Stripe
- Xero
- Capsule CRM
- JustGiving
- Oracle
Services:
- Product design
- Full-stack development
- Project management
- QA
- DevOps and CI/CD
Making demanding workloads easier to manage
The YellowDog project showed what close product and engineering collaboration can achieve when the underlying technology is complex. By combining YellowDog’s vision and market understanding with Helmes’ design, development, QA and DevOps capabilities, the teams turned an ambitious concept into a working platform that reached more than 2,500 users in its first year.
For organisations building cloud platforms, data-intensive products or software that needs to perform reliably at scale, Helmes brings together product thinking, software engineering and design to turn complex technical requirements into practical digital products.
Let’s talk.
Get in touch
