Data Engineer

2 weeks ago

Others, India Foundation for Science Innovation and Development Full time ₹ 8,00,000 - ₹ 12,00,000 per year

Scope of Work :

- Collect requirements and collaborate with city officials, agencies, system integrators, solution providers, and other data sources to integrate relevant datasets into the data exchange platform.

- Deploy and manage data exchange, while supporting application providers in building solutions for optimised management and citizen services.


Key Responsibilities :

- Evaluate APIs and datasets, create data models, develop software ETL modules, perform unit testing, and deploy them in cloud environments.

- Develop ETL modules in Python to ingest data into data exchange using REST APIs and streaming protocols such as AMQP and MQTT.

- This includes containerizing the adapters, creating data models, and catalogue entries according to data exchange specifications.

- Follow best practices for software development and adhere to Agile methodology throughout.


Required Skills :

- Excellent software design and programming skills in Python, and Java/JavaScript, with proficiency in data model technologies such as JSON-schema, JSON-LD, and Protobuf, along with strong functional testing skills.

- Solid understanding of available data sources and APIs in city solutions, including the ability to evaluate data quality, availability frequency, and common issues like data stream repetitions, as well as familiarity with various data fields and their meanings, and data structure formats (JSON, GeoJSON).

- Experience in developing ETL modules in Python for data ingestion into data exchange using REST APIs and streaming protocols such as AMQP and MQTT, including containerizing adapters and creating data models and catalogue entries according to data exchange specifications, while adhering to best practices in software development and Agile methodology.

- Good understanding and expertise in working with geospatial data (vector and raster), with experience in processing geospatial data using GIS tools like QGIS and ESRI, and skilled in analysing, transforming, and visualising geospatial data.

- Proficient in Linux, with experience in GIT version control and cloud computing platforms.

- Strong understanding of IoT, GIS, Big Data, and Cloud applications aimed at improving operational efficiency and service delivery, with a commitment to creating a positive societal impact.

- Experience in containerization using Docker, with familiarity in Kubernetes for orchestration is a plus.

- Knowledge of monitoring and logging tools such as Prometheus, Grafana, and Logstash would be a strong plus.

  • Data Engineer

    3 days ago

    Others, India Yo Hr Consultancy Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Experience : 6 to 10 Years.Qualifications : Required Qualifications : - 5 to 8 years of experience in data engineering roles, with 3+ years of hands-on experience working with Snowflake and dbt in production environments. - Proven experience building and deploying dbt models in production. - Expert SQL skills and strong understanding of ELT...

  • Senior Data Engineer

    1 day ago

    Others, India ResourceTree Global Services Pvt Ltd Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company Description : ResourceTree Global Services Pvt Ltd. (ResourceTree) is a Talent Builder focused on offering careers to professionals across various industries such as IT, Manufacturing, Services, Banking, Agri-tech, Edu-tech, and more. With over 7 years of experience, ResourceTree has helped clients hire top-tier talent by utilizing multi-pronged...

  • Senior Data Engineer

    1 day ago

    Others, India ResourceTree Global Services Pvt Ltd Full time ₹ 10,00,000 - ₹ 25,00,000 per year

    Years of Experience : 6 Job Description : We are seeking an experienced and detail-oriented Senior Data Engineer with strong expertise in QA process automation for data validation and testing. The ideal candidate will bring hands-on experience with QuerySurge and a solid foundation in ETL/ELT processes, data integration, and data quality assurance. In this...

  • Lead Data Engineer

    4 days ago

    Others, India ResourceTree Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Name : Lead Data Engineer - AWS RedShift Years of Experience : 8 Job Description : We are seeking a highly skilled and motivated Lead Data Engineer to join our Data Integration and Analytics team. The ideal candidate will bring deep expertise in AWS Redshift, with strong knowledge of Redshift architecture, query performance tuning, and workload...

  • Big Data Engineer

    1 week ago

    Others, India Unison Consulting Full time ₹ 8,00,000 - ₹ 25,00,000 per year

    Responsibilities : - Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases. - Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis. - Collaborate with data scientists, analysts, and cross-functional teams to understand business...

  • Senior Data Engineer

    3 days ago

    Others, India ResourceTree Global Services Pvt Ltd Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Job Name : Senior Data Engineer Azure Years of Experience : 5 Job Description : We are looking for a skilled and experienced Senior Azure Developer to join our team As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company. If you love learning, thinking strategically, innovating, and helping others,...

  • Senior Data Warehouse Engineer

    3 days ago

    Others, India TGS The Global Skills Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Description : The Data Warehouse Engineer will develop and maintain pipelines that transform raw ingested data into clean, analytics-ready schemas consumed by executives, managers, analysts, and clients. This is a hands-on role ideal for someone who loves working at the intersection of engineering and analytics. The candidate will join a talented...

  • Data Scientist

    1 day ago

    Others, India Xemplar Insights Full time ₹ 1,20,000 - ₹ 1,80,000 per year

    Description : Must Have Skills : - Machine learning techniques : regression, classification, clustering, deep learning. - Programming languages : Python, R. - Machine learning libraries : TensorFlow, scikit-learn, PyTorch. - Data wrangling, feature engineering. - Model validation and testing. - Cloud platforms, distributed computing, big data...

  • Senior Software Engineer/Lead

    2 weeks ago

    Others, India Crest Data Systems Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Responsibilities : - Develop, test, and maintain high-performance web applications using Angular (versions - Design and implement user interfaces with a focus on performance, scalability, and reusability. - Work closely with backend developers to integrate APIs and ensure seamless front-end functionality. - Optimize applications for maximum...

  • Data Quality Analyst

    1 week ago

    Others, India Equifax Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Data Quality Analyst Consumer. The Chief Data Office is a fast growing team critical to the Bureaus objectives of transforming into a Data and Analytics organization. It plays a critical function in building and supporting best-in-class data assets. Data is our second most important asset after our people. As such, Data Quality is a key component...