Senior Big Data Engineer (London)
Who we are
QuantumBlack teams work in multi-disciplinary environments harnessing data to provide real-world impact. We bring together the brightest data scientists, engineers and designers to take on the biggest problems facing the world’s most influential organisations. Our projects range from helping pharmaceutical companies bring lifesaving drugs to market quicker to optimising a Formula1 car’s performance. At QuantumBlack you have the best of both worlds; all the benefits of being part of one of the leading management consultancies globally and the autonomy to thrive in a fast growth tech culture.
- No project is ever the same - we work across multiple sectors, providing unique learning and development opportunities internationally.
- Fusing Tech & Leadership – We work with the latest technologies and methodologies and offer first class learning programmes at all levels
- Multidisciplinary Teamwork - Our teams include data scientists, engineers, project managers, UX and visual designers who work collaboratively to enhance performance.
- Innovative Work Culture - Creativity, insight and passion come from being balanced. We cultivate a modern work environment through an emphasis on wellness, insightful talks and training sessions.
- Striving for Diversity – With colleagues from over 40 nationalities, we recognise the benefits of working with people from all walks of life.
What we do
We guide global companies through data science solutions to transform their businesses and enhance performance across industries including healthcare, automotive, energy and elite sport.
- Healthcare Efficiency - We helped a healthcare provider improve their clinical trial practices by identifying congestion in diagnostic testing as a key indicator of admissions breaches.
- Environmental Impact – We designed and built the first data-driven application for a state of the art centre of excellence in urban innovation by collecting real-time data from environmental sensors across London and deploying proprietary analytics to find unexpected patterns in air pollution.
- Product Development - We worked with the CEO of an elite automotive organisation to reduce the 18-month car development timeframe by improving processes, designs and team structures.
Who you are
A core value at QuantumBlack is fusion and at the heart of our multi-disciplinary teams is the belief that the sum of individual parts will always be less than the impact of the entire team. You are a highly collaborative individual who is capable of laying aside your own agenda, listening to and learning from colleagues, challenging thoughtfully and prioritising impact. You search for ways to improve things and work collaboratively with colleagues. You believe in iterative change, experimenting with new approaches, learning and improving to move forward quickly. Trust between colleagues is paramount here – you are an individual who can always be trusted to work in the best interests of all colleagues and to achieve the best outcome for QuantumBlack and our clients. You are naturally enthusiastic and enjoy sharing your passion with others.
Who You’ll work with:
Our Consultant Data Engineers work closely with our clients and our Data Scientists in order to curate, transform and construct features which feed directly into our modelling approach.
This would be a hybrid client-facing/technical role using cutting edge technologies, whilst also being able to communicate complex intractable ideas to non-technical audiences. Gathering clear requirements is a key part of this role and will define the technical strategy the team employs on the study.
What You'll Do
- Work with our clients to model their data landscape, obtain data extracts and define secure data exchange approaches
- Acquire, ingest, and process data from multiple sources and systems into Big Data platforms
- Understanding, assessing and mapping the data landscape.
- Maintaining our Information Security standards on the engagement.
- Collaborate with our data scientists to map data fields to hypotheses and curate, wrangle, and prepare data for use in their advanced analytical models
- Defining the technology stack to be provisioned by our infrastructure team.
- Building modular pipeline to construct features and modelling tables.
- Use new and innovative techniques to deliver impact for our clients as well as internal R&D projects
- Mentoring and developing junior Data Engineers on engagements
- Strong experience with at least two of the following technologies: Python, Scala, SQL, Java
- Commercial client-facing project experience is beneficial, including working in close-knit teams
- The ability to work across structured, semi-structured, and unstructured data, extracting information and identifying linkages across disparate data sets
- Good experience in multiple database technologies such as:
- Distributed Processing (Spark, Hadoop, EMR)
- Traditional RDBMS (MS SQL Server, Oracle, MySQL, PostgreSQL)
- MPP (AWS Redshift, Teradata)
- NoSQL (MongoDB, DynamoDB, Cassandra, Neo4J, Titan)
- A proven ability in clearly communicating complex solutions
- Have a strong understanding of Information Security principles to ensure compliant handling and management of client data
- Experience and interest in Cloud platforms such as: AWS, Azure, Goole Platform or Databricks
- Strong experience in traditional data warehousing / ETL tools (Informatica, Talend, Pentaho, DataStage)
- Exceptional attention to detail
- Recruiter screen
- Online technical test
- Technical case study interview
- Final technical discussion and interview
- Technical and non-technical training courses
- Attendance at conferences
- McKinsey benefits (competitive salary, annual bonus, generous pension scheme etc.)
- Free lunches daily
- Free art pass
- Gym membership and yoga classes