Connecting...

BIG DATA ENGINEER

  • Location:

    Dublin City Centre

  • Contact email:

    ckelly@sigmar.ie

  • Sector:

    IT

  • Contact phone:

    01 4744637

  • Job type:

    Full Time

  • Job ref:

    ClaireBDACK

  • Salary:

    Dependent on Experience

  • Published:

    13 days ago

  • Contact email:

    ckelly@sigmar.ie

  • Consultant:

    Claire Kelly

Job description

Great role available for an experienced Big Data Engineer within a global Fintech company based in Dublin City Centre

What you will do:

This role will design and develop data pipelines, enabling the movement of data into the company's global Enterprise Analytics Platform.

Responsibilities:

• Work with the Data Engineering Manager to define and deliver data solutions

Design, develop, test, tune and deploy advanced data integration pipeline solutions,

• Work with key business stakeholders to gather data integration requirements and develop solutions accordingly.

• Contribute to the logical/physical design of the data lake and the development of all data assets, and downstream warehouses/marts.

• Maintain up to date Technical and Operational documentation.

• Manage task list and communicate updates to stakeholders and team members following an Agile Scrum methodology.

What you will need:

Hands-on experience developing data pipelines within a Hadoop-based or Apache Spark environment (solution stack is preferably Cloudera, but similar stacks considered).

Understanding of modern software engineering tools such as Git, Bitbucket, TeamCity, Jenkins, Maven, Octopus

Good knowledge of Hadoop ecosystem (HDFS, Hive, Impala, Kafka, Spark, ElasticSearch etc.), and associated tools and cloud-based technologies (AWS-based).

Experience in programming (Python, Scala, Java) for automation and data processing.

Highly proficient at reading, profiling, parsing, transforming, cleansing and integrating data from various sources (structured and unstructured).

Experience developing data solutions using native IaaS and PaaS solutions on AWS (EMR, Redshift, RDS, S3) a distinct advantage.

Distributed Computing and/or Functional programming . Experience programming Apache Spark solutions (PySpark, Scala, Java, R) etc. a distinct advantage.

Knowledge of Agile Scrum and Continuous Delivery practices is desirable

Basic knowledge of Machine Learning algorithms and techniques a plus.

To apply for this role or for more information, contact Claire on ckelly@sigmar.ie or call +35314744637 

Following your application for this specific role, Sigmar may contact you regarding other positions that we feel you may be suitable for. If you do not wish to be contacted about other opportunities please let us know. For further information please refer to the Privacy Statement on our website.