Tx Data Ingestion

2 weeks ago

Chennai, India Encora Full time

**Our company**:
Encora is a global Software and Digital Engineering company that helps business overcome the Software Engineering Talent shortage and provides next-gen services and such as Predictive Analysis, Artificial Intelligence & Machine Learning, IoT, Cloud, and Test Automation. We count with 16 global offices and 25 innovation labs.

Our Software Engineering experts work with leading-edge technology companies to improve their speed to impact.

**About the role**:
**Task Description**: Programming languages: Python (main), Scala (plus)

**Batch Data Ingestion**: ADF and/or Spark, Delta IO (plus), Iceberg (plus plus)

**Streaming Data Ingestion**: Kafka and Spark Streaming

**Binary Formats**: Parquet and Avro (main), ORC (plus)

**Warehousing**: Snowflake (main), Synapse or others (plus)

**No-SQL**: Column store (plus) (e.g. Cassandara, HBase or equivalent), Document store (plus) (e.g. Mondo DB, Couchdb or equivalent), Key/Value store (plus) (e.g. Redis, RocksDB or equivalent)

**Architectural patterns**: Data Vault Modelling (main), Lakehouse (main), Semantic Versioning and Schema Evolution (nice to have), Data Meshing (main), CAP theorem (plus), Semantic delivery reliability of distributed systems (plus, expected in streaming use case scenarios), Microservices (plus), Threading models (e.g. Threads, Actors, Co-routines) (plus)

**CI/CD**: Azure Devops (main), Jenkins, Git Actions, etc. (plus)

**Bonus**: Docker and Kubernetes

**Work Result**: Code submitted to repository

**Skill Area**: Other

**Technology**: Other

**Proficiency - Technology**: Expert

**Other Skills**: Azure Data

  • Data Engineer

    1 month ago

    Hyderabad/Chennai/Pune, IN Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...

  • Data Engineer

    1 month ago

    Hyderabad/Chennai/Pune, India Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...

  • Data Engineer

    3 weeks ago

    Hyderabad/Chennai/Pune, IN Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...

  • Data Engineer

    3 weeks ago

    Hyderabad/Chennai/Pune, India Freelancer HR Full time

    Description :Job Description :- Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Data bricks and/or HDInsights- Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)- Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB,...

  • Azure Architect

    3 weeks ago

    Delhi NCR/Bangalore/Pune/Chennai/Mumbai/Hyderabad, India ACZ Global Private Limited Full time

    Experience Needed: - 9 -14 years of industry experience and at least 3 years of experience in architect role is required along with at least 3 to 4 years' experience designing and building analytics solutions in Azure. - Experience in architecting data ingestion/integration frameworks capable of processing structured, semi-structured & unstructured data...

  • Azure Architect

    2 months ago

    Delhi NCR/Bangalore/Pune/Chennai/Mumbai/Hyderabad, India ACZ Global Private Limited Full time

    Experience Needed: - 9 -14 years of industry experience and at least 3 years of experience in architect role is required along with at least 3 to 4 years' experience designing and building analytics solutions in Azure. - Experience in architecting data ingestion/integration frameworks capable of processing structured, semi-structured & unstructured data...

  • Azure Architect

    2 months ago

    Delhi NCR/Bangalore/Pune/Chennai/Mumbai/Hyderabad, IN ACZ Global Private Limited Full time

    Experience Needed: - 9 -14 years of industry experience and at least 3 years of experience in architect role is required along with at least 3 to 4 years' experience designing and building analytics solutions in Azure. - Experience in architecting data ingestion/integration frameworks capable of processing structured, semi-structured & unstructured data...

  • Data Visualization

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Must Have BI Tool: Quick Sight (2 years ), other BI Tools ( 5 years) ,SQL ScriptingCloud : RDS,S3, IAM, Pipeline monitoring, Security, Logging and monitoring Services. Database : RDS,SQL, Athena, Flat file Devops Tools : Terraform ,GIT,GIT actions, CICD...

  • Data Visualization

    2 weeks ago

    Chennai, India NTT DATA Full time

    Job DescriptionNTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Must HaveBI Tool: Quick Sight...

  • Data Visualization

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). **Must Have** - BI Tool: Quick Sight (2 years ), other BI Tools ( 5 years) ,SQL Scripting Cloud : RDS,S3, IAM, Pipeline monitoring, Security, Logging and monitoring Services. - Database : RDS,SQL, Athena, Flat file - Devops Tools : Terraform ,GIT,GIT...

  • Data Visualization

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time

    Job DescriptionNTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Must HaveBI Tool: Quick Sight...

  • Data Visualization

    2 weeks ago

    Chennai, India NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Must HaveBI Tool: Quick Sight...

  • Data Visualization

    2 weeks ago

    chennai, India NTT DATA Full time

    Job Description NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). Must Have BI Tool:...

  • Data Handling

    2 months ago

    Chennai, India Elitez Recruitment Services Private Limited Full time

    **Roles and Responsibilities**: Prerequisites**:Proficient with PYTHON to handle Data Structures & JSON, CSV file formats**. Comfortable in scripting using **Python**. Bonus: Understands or have experience in **web** **scraping** Responsibilities: Receive & Process data from Internal tools, clean or enrich the data to suit the Deliverables. Update the...

  • Data Handling

    2 days ago

    Chennai, India Elitez Recruitment Services Private Limited Full time

    **Roles and Responsibilities**: Prerequisites**:Proficient with PYTHON to handle Data Structures & JSON, CSV file formats**. Comfortable in scripting using **Python**. Bonus: Understands or have experience in **web** **scraping** Responsibilities: Receive & Process data from Internal tools, clean or enrich the data to suit the Deliverables. Update the...

  • Data Visualization

    6 days ago

    Chennai, Tamil Nadu, India NTT DATA Services Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).Must HaveBI Tool: Quick Sight (2 years ),...

  • Data Visualization

    2 weeks ago

    chennai, India NTT DATA Services Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).   Must Have BI Tool: Quick Sight (2 years...

  • Data Visualization

    2 weeks ago

    Chennai, India NTT DATA Services Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). Must HaveBI Tool: Quick Sight (2 years ),...

  • Data Visualization

    2 weeks ago

    Chennai, India NTT DATA Services Full time

    NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Visualization to join our team in Chennai, Tamil Nādu (IN-TN), India (IN).   Must Have BI Tool:...

  • Data Architect

    6 days ago

    Chennai, Tamil Nadu, India hubwork Full time

    Qualifications : 10 to 15 years of experience in Data Architect with solid experience in design, development and implementation of endtoend data ingestion and data processing system in AWS platform. 23 years of experience in AWS Glue, Lambda, Appflow, EventBridge, Python, PySpark,Bigdata,Spark Lake House, S3, Redshift, Postgres, API Gateway, CloudFormation,...