Back to jobs
Big Data Engineer (Google Cloud)
- Posted 18 October 2023
- Salary Negotiable
- LocationUnited States Virgin Island
- Job type Contract
- Discipline Technology
- ReferenceBBBH8109_1697666716
- Contact NameSarah Ford
Job description
Our client, a well-known provider of diverse financial solutions and innovative services, is in search of a Big Data Engineer with expertise in Google Cloud. The ideal candidate will excel in cloud software development and big data technologies, optimizing and managing data pipelines within a collaborative team environment.
Must Have Qualifications:
- 5+ years of Cloud software development experience
- 3-5 years of solid experience with GCP BigQuery
- 3 years of experience working on GCP Compute Engine
- 5+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL, and PySpark)
- Good overall understanding of Google Cloud Platform and its services
- Expertise with data structures, data modeling, and software architecture
- 3-5 years solid experience with Google Cloud Dataproc
Preferred Qualifications:
- Ability to design and develop optimized data pipelines for batch and real-time data processing
- Experience with GitHub and leveraging CI/CD pipelines
- Experience in Hadoop and Spark Architecture and its working principle
- Experience with NoSQL i.e., HBase, Couchbase, MongoDB
- Experience in UNIX shell scripting
- Hands-on experience on writing and understanding complex SQL (Hive/PySpark-dataframes), optimizing joins while processing a huge amount of data
- Experience in designing and building solutions using Kafka streams or queues
- Should have experience in analysis, design, development, testing, and implementation of system applications
- Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows
- Knowledge of the financial reporting ecosystem will be a plus
Preferred Location: Phoenix, AZ (Remote Work Considered)
![](https://counter.adcourier.com/c2FyYWguZm9yZC44MzE5Mi43MDY3QG1hcmtqYW1lcy5hcGxpdHJhay5jb20.gif)