3 Hdfs jobs in India
Principal Engineer – Ozone/HDFS
Posted today
Job Viewed
Job Description
Job Description
:At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.
Cloudera is looking for an exceptional and passionate software engineer with a strong distributed systems background to join the Storage Engineering team focused on building Apache Ozone. The Storage team is responsible for primary storage and storage access layers, which are core to the platform. Apache Ozone () provides a massively scalable distributed object store with a distributed file system interface. Ozone is designed to scale to tens of billions of files and blocks, and overcome the limitations of Hadoop Distributed File System (HDFS), namely, millions of small files and managing a huge number of datanodes.
Opportunity to join the team that created and wrote most of the HDFS code and make a huge impact on the big data and cloud computing industry.
As a Principal Software Engineer, you will:
You will be directly involved in the design and implementation of the core feature set of Apache Ozone and Apache Ratis (open-source RAFT implementation)
You will regularly contribute code and design docs to the Apache open-source community.
As part of storage engineering, you will support enterprise customers running 100s of petabytes-scale big data analytics and ML/AI pipelines.
You will partner with Engineering leaders, product managers, and cross-functional teams as a part of the Cloudera Data platform ecosystem in understanding requirements and turning them into a solid design and implementation, and facilitating integration and adoption.
Additionally, in this role, you will be responsible for leading a talented group of engineers working on the feature and mentoring junior engineers.
We are excited about you if you have:
BS, MS, or PhD in Computer Science
Bachelor's +15 , Master's +12 years of relevant industry experience required (8+ for PhD candidate)
Strong backend engineering skill set with expertise in Java, or strong C++ skills, with intermediate Java expertise
Passionate about programming. Clean coding habits, attention to detail, and focus on quality
Experience with large-scale, distributed systems design and development with a strong understanding of scaling, replication, consistency, and high availability
Solid experience with system software design and development with a strong understanding of computer architecture, storage, network, and IO subsystems, and distributed systems
Hands-on programmer with strong data structures and algorithms skillset
Strong oral and written communication skills
You may also have:
Strong background in a distributed storage system, including file systems, database storage internals, NoSQL storage, or distributed hash tables
Strong background in performance tuning, identifying performance bottlenecks, and implementing performance optimizations
Strong understanding of the Apache Big Data ecosystem and over 3+ years of experience in systems software, including file systems
Recognized contributions to open source projects
Experience using projects such as Hive, Pig, MapReduce, HBase, etc., is a big plus
Good Understanding of storage development, RAFT replication framework, or equivalent distributed consensus frameworks
What you can expect from us:
Generous PTO Policy
Support work life balance with
Flexible WFH Policy
Mental & Physical Wellness programs
Phone and Internet Reimbursement program
Access to Continued Career Development
Comprehensive Benefits and Competitive Packages
Employee Resource Groups
Data Analyst (SQL,HDFS, Hive)
Posted today
Job Viewed
Job Description
RiskInsight Consulting Pvt Ltd is looking for a Data Analyst with expertise in SQL and HDFS (Hadoop Distributed File System) to join our dynamic team. The selected candidate will play a crucial role in analyzing large datasets and providing actionable insights to drive business decisions. If you are a detail-oriented individual with a passion for data, we want to hear from you!
- Design, implement, and maintain data pipelines and workflows using HDFS for data management.
- Conduct data extraction, transformation, and loading (ETL) processes using SQL and Hive.
- Perform rigorous data analysis to identify trends, inconsistencies, and insights for stakeholders.
- Collaborate with business units to understand their analytical needs and deliver data-driven insights.
- Create visualizations and generate reports to present findings to both technical and non-technical audiences.
- Ensure data integrity and security throughout all stages of data processing.
- Stay informed about industry advancements in data analysis and contribute to best practices in the field.
Requirements
- Bachelor’s degree in Computer Science, Data Science, Mathematics, or related field.
- Proven experience in a Data Analyst role with strong expertise in SQL and HDFS.
- Hands-on experience with Hive for querying and analyzing large datasets.
- Familiarity with data visualization tools such as Tableau, Power BI, or similar.
- Excellent problem-solving and analytical skills with the ability to draw meaningful conclusions from data.
- Strong communication skills to effectively share insights with diverse teams.
- Ability to work independently and as part of a team in a fast-paced environment.
Benefits
Competitive salary and benefits package.
Opportunity to work on cutting-edge technologies and solve complex challenges.
Dynamic and collaborative work environment with opportunities for growth and career advancement.
Regular training and professional development opportunities.
Java, Spark - Lead Developer (Java, Spark, HDFS, Hive, Hadoop) - Vice President

Posted 15 days ago
Job Viewed
Job Description
**Responsibilities:**
+ Partner with multiple management teams to ensure appropriate integration of functions to meet goals as well as identify and define necessary system enhancements to deploy new products and process improvements
+ Resolve variety of high impact problems/projects through in-depth evaluation of complex business processes, system processes, and industry standards
+ Provide expertise in area and advanced knowledge of applications programming and ensure application design adheres to the overall architecture blueprint
+ Utilize advanced knowledge of system flow and develop standards for coding, testing, debugging, and implementation
+ Develop comprehensive knowledge of how areas of business, such as architecture and infrastructure, integrate to accomplish business goals
+ Provide in-depth analysis with interpretive thinking to define issues and develop innovative solutions
+ Serve as advisor or coach to mid-level developers and analysts, allocating work as necessary
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ **10+ years of relevant experience in Apps Development or systems analysis role using Java, Spark, HDFS, Hive, Hadoop**
+ **i)Hands-on development expertise in Java with Spark.** **Extensive knowledge of HDFS, Hive(Relevant experience of around4-5 years .** **ii) Hands on knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns.** **iii) Familiarity with data formats like Avro, Parquet, CSV, JSON.** **iv) Java knowledge with advanced skills in multithreading and multiprocessing, along with extensive experience in efficiently processing large-scale data.**
+ Extensive experience system analysis and in programming of software applications.
+ Experience in managing and implementing successful projects.
+ Subject Matter Expert (SME) in at least one area of Applications Development.
+ Ability to adjust priorities quickly as circumstances dictate.
+ Demonstrated leadership and project management skills.
+ Consistently demonstrates clear and concise written and verbal communication.
**Skills required:**
+ **Highly experienced and skilled Java technical lead with 10-12 years of experience with software building and platform engineering.**
+ **Extensive development expertise in building the high scaled and performant software platforms for data computation and processing.**
+ **Expert level knowledge of core Java concepts and framework such as Spring Boot, Microservices and well versed with OOPs concepts and design patterns.**
+ **Java expert with advanced skills in multithreading and multiprocessing, along with extensive experience in efficiently processing large-scale data.**
+ **Expertise and hands-on experience on working with Apache Spark using Java and understanding of the Bigdata ecosystem and design principles**
+ **Hands-on experience on Unix and python/shell scripting.**
+ **Good knowledge in Hadoop, YARN, Hive, Spark, and Spark SQL with extensive high volume of data processing pipeline development.**
+ **Strong computer science fundamentals in data structures, algorithms, databases, and operating systems.**
+ **Highly experienced with Unix based operating systems and shell scripting.**
+ **Strong analytical and logical skills.**
+ **Hands-on experience in writing SQL queries.**
+ **Experience with source code management tools such as Bitbucket, Git etc.**
+ **Experience working with banking domain like pricing, risk etc. is plus.**
+ **Consistently demonstrates clear and concise written and verbal communication**
**Education:**
+ Bachelor's degree/University degree or equivalent experience
+ Master's degree preferred
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
Be The First To Know
About the latest Hdfs Jobs in India !