Data Engineer

IT & Technology
Data Engineer

Our client in London a brokerage firm are seeking a Data Engineer to work on the firm's Big Data Analytics platform with a specialism in Java.

The successful candidate will work with the data team to build out the existing platform to support additional technology and business use-cases involving both on-premise and Cloud implementations. The platform delivers data solutions across the group including use-cases for: market data analytics, regulatory reporting, surveillance and revenue analysis.

Key responsibilities:

  • To work in partnership with the business/business analysts to identify key requirements for implementation
  • To identify any technical requirements for new products
  • To work with all the business and technology departments to ensure all business and technical requirements are met
  • To identify and manage any integration work
  • To analyse, design and build any such projects
  • To provide input into the current and on-going system architecture
  • To liaise with other development teams as necessary to implement cross-team projects
  • Be alert to Conduct Risk issues, specifically the risk of harm to client interests, market integrity and/or competition in financial markets due to inappropriate practices or behaviours across the firm

To undertake and manage:

  • Systems analysis and design
  • Systems development
  • Systems documentation
  • Production support and out-of-hours system maintenance

Skills / experience required:

  • Strong Data background. Ideally worked previously on clustered data lakes with distributed computing.
  • Expertise with Java. Including multithreading, profiling, performance tuning and monitoring.
  • Solid database experience. Relational and NoSQL, including logical and physical schema modelling, performance optimisation and management.
  • Experience with streaming data - demonstrable in-depth understanding of at least one messaging broker - including performance and monitoring.
  • Experience with ETL approach including automation/scheduling - prove in-depth knowledge of at least one frameworks or tool.
  • Linux/Unix (5 years) - including some shell scripting and understanding of repositories and resources (cpu, mem, disk, network)
  • Strong CI/CD focus

Additional Skills / experience:

  • AWS: Data Pipeline, S3, EMR, Lamda, DynamoDB, RedShift
  • Hadoop and BigData stack - including Yarn, HDFS, Kafka and Spark.
  • Some DevOps experience - deploying and automating. Possibly used one scripting language, puppet, chef, ansible, terraform.
  • Containerization technologies - e.g. Docker, K8s
  • Monitoring frameworks/tools
  • Knowledge of Python is beneficial
  • Experience of software development in a financial services environment advantageous
  • Willingness to keep up to date with latest technology trends and proactively identify appropriate areas into which they can be applied.
  • Solid Computing Degree

Personal attributes:

  • Team player with excellent inter-personal skills and confident communication skills.
  • Able to effectively disseminate knowledge and experience to less experienced team members.
  • Must be able to deal with our customers effectively, ie development teams working across the globe using Messaging and Data products, alongside external vendors who use the products.
  • Must be able to deal with and adapt to change extremely effectively.
  • Must be proactive in generating ideas and effective at developing solutions that are balanced, proportionate and effective. It is critical that this is achieved in collaboration with the global team.
  • Ability to work and function under pressure, handle multiple tasks, and shifting priorities
  • “Self-starter” always looking to improve quality of process and deliverables and keen to take a lead role in that process

Can't find the job you're looking for, send us your info and we will review your options?

(Permitted file size is 5Mb and file types are: doc, docx, txt, pdf, rtf, xls)

Please complete the form below, your data is processed here under legitimate interest. Please also note that for your privacy no data from this form is stored in this website (view our privacy policy).

Attach CV*