Senior Data Engineer

2 weeks ago

India 3Pillar Global Full time
Job Description

Embark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making.

As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
If your passion for data engineering solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering

Responsibilities:

- Lead the design and implementation of scalable ETL/data pipelines using Python and Luigi for data processing.
- Ensure efficient data processing for high-volume clickstream, demographics, and business data.
- Guide the team in adopting best practices for data pipeline development, code quality, and performance optimization.
- Configure, deploy, and maintain AWS infrastructure, primarily AWS EC2, S3, RDS, and EMR, to ensure scalability, availability, and security.
- Support data storage and retrieval workflows using S3 and SQL-based storage solutions.
- Provide architectural guidance for cloud-native data solutions and infrastructure design.
- Oversee legacy framework maintenance, identify improvement areas, and propose comprehensive cloud migration or modernization plans.
- Lead the strategic planning, execution, and optimization of large-scale data migration initiatives to Snowflake, ensuring data integrity, security, and minimal business disruption.
- Coordinate infrastructure changes with stakeholders to align with business needs and budget constraints.
- Develop and implement robust monitoring solutions to track system health, performance, and data pipeline accuracy.
- Set up alerts and dashboards for proactive issue detection and collaborate with cross-functional teams to resolve critical issues.
- Lead efforts in incident response, root cause analysis, and post-mortem processes for complex data engineering challenges.
- Document workflows, troubleshooting procedures, and code for system transparency and continuity.
- Provide mentoring and training to team members on best practices and technical skills.
- Foster a culture of continuous learning, knowledge sharing, and technical excellence within the data engineering team.

Qualification

- Experience: 6 years of experience in data engineering or a related technical field, with at least 4 years in a Snowflake projects.
- Cloud Data Warehousing: Proven expertise with Snowflake data warehousing, including schema design, efficient data loading (e.g., Snowpipe, COPY into), performance tuning, and robust access control mechanisms.
- Programming & Scripting: Strong programming skills in Python for automation and data workflows.
- Data Processing & Storage: Expertise in managing SQL databases for data storage and query optimization.
- Monitoring & Alerting Tools: Familiarity with monitoring solutions for real-time tracking and troubleshooting of data pipelines.
- Experience with the AWS Cloud and data engineering services

  • Senior Data Engineer

    3 weeks ago

    India Aptus Data Labs Full time

    Job Title: Senior Data EngineerLocation: RemoteExperience: 4+Employment Type: Full-TimeNotice- Looking for immediate joiners or who can join within 15-20 days.About the RoleAptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. You'll work on modern data lake...

  • Senior Data Engineer

    4 weeks ago

    India Asenium Consulting Full time

    For one of my customers, we are looking for a Senior Data Engineer.Location: India (100% remote)Duration: 6+ months (renewable)Description: We are looking for a Senior Data Engineer with strong experience in SQL, Talend, and cloud platforms such as Google Cloud (Big Query) and Microsoft Azure.You will be responsible for designing and managing ETL pipelines,...

  • Senior Data Engineer

    4 weeks ago

    India Black Piano Full time

    OverviewAs a Senior Data Engineer, you'll play a crucial role in storing, processing, modelling, and applying data science to make data and insights available for analytics and business intelligence (BI) systems. This position offers a unique opportunity to work with cutting-edge products and world-class clients in a remote-first environment.Key...

  • Data Engineer

    4 weeks ago

    India Indsafri Full time

    Hiring Now: Senior Data Engineers (Banking Domain – 2 Positions)Location: RemoteExperience: 7+ Years | Banking sector experience preferred Type: Contract About the OpportunityWe're looking for highly skilled Senior Data Engineers to join a mission-critical banking project. This is your chance to work on cutting-edge data pipelines, streaming platforms,...

  • Data Engineer

    2 days ago

    India Insight Global Full time

    100% Remote Data EngineerRequired Skills & Experience5+ years of experience as a Data EngineerExpertise in Python, Pyspark and SQLHands-on Python/PySpark coding experienceStrong Big Data experience, GCP preferredTelemetry experienceStreaming experience with analytics and data processingBusiness analytics experience and working directly with business partners...

  • Senior Data Engineer – GCP

    4 weeks ago

    India Xebia Full time

    We're Hiring: Senior Data Engineer – GCP | Databricks | E-commerce Domain Work Locations: Chennai | Bangalore | Hyderabad | Gurugram | Jaipur | Pune | Bhopal Experience: 7–8+ Years Shift Timing: 2 PM – 10 PM IST Work Mode: Hybrid – 3 days/week from office Only Immediate Joiners (0–15 Days Notice)Are you a seasoned Data Engineer passionate about...

  • TAVS Senior Data Engineer(Location: Pune)

    4 weeks ago

    India DigiHelic Solutions Pvt. Ltd. Full time

    TAVS Senior Data EngineerLocation: PuneExperience: 12 Years1. Proficiency in PostgreSQL, including database design, optimization, and management.2. Extensive experience with Databricks for data processing and analytics, and/or any other Apache Spark distribution.3. Strong programming skills in Python, especially with libraries such as Pandas and NumPy.4....

  • Senior Data Engineer

    2 days ago

    India BrightEdge Full time

    Big Data Engineer - BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us...

  • TAVS Senior Data Engineer(Location: Pune)

    4 weeks ago

    India DigiHelic Solutions Pvt. Ltd. Full time

    TAVS Senior Data EngineerLocation: PuneExperience: 12 Years1. Proficiency in PostgreSQL, including database design, optimization, and management. 2. Extensive experience with Databricks for data processing and analytics, and/or any other Apache Spark distribution. 3. Strong programming skills in Python, especially with libraries such as Pandas and NumPy. 4....

  • Senior Software Engineer – Data Systems

    21 hours ago

    India beBeeSoftwareEngineer Full time ₹ 1,74,27,200 - ₹ 2,51,04,000

    As a senior software engineer, you will focus on developing efficient and scalable applications that process large-scale manufacturing data from globally distributed plants. Your key responsibilities include building data-driven solutions using IoT devices, machine learning models, and advanced data processing techniques.The goal is to create highly...