Big Data and Gcp

4 days ago

Bengaluru Karnataka, India Manage Captive Solutions Full time

**Who We Are**
Manage Captive Solutions (MCS) stands as the ultimate solution for effortless offshore hiring, precise payroll management, streamlined administration, and stringent compliances. We provide businesses distinctive solutions, making it easy to hire and onboard new team members. By removing the hurdles of setting up a local office, we enable businesses to focus on expanding their horizons.Our holistic service oversees the entire onboarding process, managing everything from local regulations to payroll intricacies and compliance matters. Additionally, our commitment to cutting-edge technology delivery further enhances their business operations.

**Job Title**: Big Data and GCP
**Experience Level**: 4-8 years
**Location**: Bangalore (On-site)
**Availability**: Immediate

Key Responsibilities:

- Design, develop, and maintain scalable data pipelines and ETL processes using GCP services such as Dataflow, Dataprep, and Composer.
- Implement data ingestion, transformation, and storage solutions using technologies like Hive, Spark, PySpark, and Python.
- Work closely with cross-functional teams to understand data requirements and translate them into technical solutions.
- Optimize and tune data pipelines for performance, reliability, and efficiency.
- Collaborate with data scientists and analysts to provide access to high-quality data and ensure data integrity.
- Monitor, troubleshoot, and resolve issues related to data processing and performance.
- Stay updated with emerging technologies and industry trends in big data and cloud computing.

Qualifications:

- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 4 to 8 years of experience in big data engineering and cloud technologies, with a focus on GCP.
- Strong proficiency in programming languages such as Python and SQL.
- Hands-on experience with big data processing frameworks like Spark and Hive.
- Experience with GCP services including BigQuery, Pub/Sub, Dataflow, and Composer.
- Knowledge of CI/CD practices and tools for automating deployment and testing.
- Familiarity with database systems such as MySQL, NoSQL, and Google Cloud SQL.
- Experience with real-time data streaming technologies like Kafka is a plus.
- Excellent communication skills and ability to work effectively in a collaborative team environment.

**How to Apply**:
Manage Captive Solutions is an equal opportunity employer and encourages individuals from diverse backgrounds to apply.

**Job Types**: Full-time, Permanent

**Benefits**:

- Health insurance
- Provident Fund

Schedule:

- Day shift
- Monday to Friday

**Experience**:

- total work: 5 years (preferred)

Ability to Commute:

- Bangalore, Karnataka (required)

Ability to Relocate:

- Bangalore, Karnataka: Relocate before starting work (required)

Work Location: In person

**Speak with the employer**

+91 7895546435

  • GCP Big Data Developer

    2 days ago

    Bengaluru, India Impetus Full time

    ```html About the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    7 hours ago

    Bengaluru, India Impetus Full time

    ```htmlAbout the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    2 weeks ago

    Bengaluru, India Impetus Full time

    ```html About the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    1 week ago

    Bengaluru, India Impetus Full time

    ```html About the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    2 weeks ago

    Bengaluru, India Impetus Full time

    ```html About the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    1 week ago

    Bengaluru, India Impetus Full time

    ```htmlAbout the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).Responsibilities- Should be aware with columnar database e.g. parquet, ORC etc.-...

  • GCP Big Data Developer

    1 week ago

    Bengaluru, India Impetus Full time

    ```htmlAbout the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).Responsibilities- Should be aware with columnar database e.g. parquet, ORC etc.-...

  • Gcp big data developer

    1 day ago

    Bengaluru, India Impetus Full time

    ```htmlAbout the CompanyMust have: Bigdata, GCP, Dataproc, Dataflow, SQL, SparkAbout the RoleMust have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services).ResponsibilitiesShould be aware with columnar database e.g. parquet, ORC etc.Visualize...

  • GCP Big Data Developer

    1 week ago

    Bengaluru, India Impetus Full time

    ```html About the Company Must have: Bigdata, GCP, Dataproc, Dataflow, SQL, Spark About the Role Must have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services). Responsibilities - Should be aware with columnar database e.g. parquet, ORC etc. -...

  • GCP Big Data Developer

    1 week ago

    Bengaluru, India Impetus Full time

    ```html About the Company Must have: Bigdata, GCP, Dataproc, Dataflow, SQL, Spark About the Role Must have: Operating knowledge of cloud computing platforms (GCP, especially Big Query, Dataflow, Dataproc, Storage, VMs, Networking, Pub Sub, Cloud Functions, Composer services). Responsibilities Should be aware with columnar database e.g. parquet, ORC etc....