Big Data Engineer (Google Cloud)

Posted 18 October 2023
Salary Negotiable
Job type Contract
Discipline Technology
Contact NameSarah Ford

Job description

Our client, a well-known provider of diverse financial solutions and innovative services, is in search of a Big Data Engineer with expertise in Google Cloud. The ideal candidate will excel in cloud software development and big data technologies, optimizing and managing data pipelines within a collaborative team environment.

Must Have Qualifications:

  • 5+ years of Cloud software development experience
  • 3-5 years of solid experience with GCP BigQuery
  • 3 years of experience working on GCP Compute Engine
  • 5+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL, and PySpark)
  • Good overall understanding of Google Cloud Platform and its services
  • Expertise with data structures, data modeling, and software architecture
  • 3-5 years solid experience with Google Cloud Dataproc

Preferred Qualifications:

  • Ability to design and develop optimized data pipelines for batch and real-time data processing
  • Experience with GitHub and leveraging CI/CD pipelines
  • Experience in Hadoop and Spark Architecture and its working principle
  • Experience with NoSQL i.e., HBase, Couchbase, MongoDB
  • Experience in UNIX shell scripting
  • Hands-on experience on writing and understanding complex SQL (Hive/PySpark-dataframes), optimizing joins while processing a huge amount of data
  • Experience in designing and building solutions using Kafka streams or queues
  • Should have experience in analysis, design, development, testing, and implementation of system applications
  • Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows
  • Knowledge of the financial reporting ecosystem will be a plus

Preferred Location: Phoenix, AZ (Remote Work Considered)