Bigdata Architect

10-12 years of deep technical experience in data engineering, primarily focusing on high-tech or cloud-based environments.
- Extensive expertise in building, maintaining, and optimizing large-scale data platforms and pipelines. Must have knowledge of Big Data Architecture Patterns and experience in the delivery of end-to-end Big Data solutions.
- Advanced hands-on experience with:

- ETL/ELT frameworks: (e.g., Apache Nifi, Airflow, dbt)
- Data Warehousing: (e.g., Snowflake, Redshift, BigQuery)
- Big Data Technologies: (e.g., Hadoop, Apache Spark, HDFS)
- Streaming Data Platforms: (e.g., Apache Kafka, Kinesis, Flink)
- Cloud Data Solutions: (AWS, GCP) with a focus on services like Lambda, EMR, Glue, and DataFlow.
- Have 8+ year of experience in ClickHouse/Redshift or any other OLAP system
- Proficient in data processing and programming with Python, Scala, or Java, leveraging these languages for building high-performance data pipelines.
- Strong understanding of distributed computing, microservices architecture, and containerization technologies (e.g., Kubernetes, Docker).
- Deep knowledge of data security, data governance, and compliance frameworks in large-scale, complex data ecosystems.
- Experience in performance optimization and troubleshooting in high
- throughput data environments.
- worked in Hadoop eco-system with AWS cloud distribution and worked at least on one or more big data ingestion tools (Sqoop, Flume, NiFI, etc), distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub, etc).

**Job Types**: Contractual / Temporary, Freelance
Contract length: 6 months

Pay: ₹400,352.44 - ₹2,000,000.00 per year

**Benefits**:

- Work from home

Schedule:

- Day shift
- Fixed shift
- Monday to Friday

Application Question(s):

- How soon you can join?

**Experience**:

- Spark steaming: 3 years (required)
- Big Data: 10 years (required)

Work Location: Remote