Software Engineer (Platform)
Needl.ai is hiring a Software Engineer (Platform) to build the pipeline that turns web pages, documents, filings and feeds into structured, searchable, AI-ready knowledge.
Apply now
Needl.ai turns messy real-world data (public web pages, private documents, filings, feeds) into structured, searchable, AI-ready knowledge. You'll work on the pipeline that makes that happen: fetching data from sources, pulling it in and parsing it, cleaning and enriching it, and preparing it so it can be consumed downstream by search and LLM-powered features.
What you'll do
- Build and maintain
- LLM-powered crawlers for public and private sources: web scraping, APIs, authenticated flows, and detecting when a source changes.
- Unstructured text extraction from documents, using everything from basic techniques and libraries to advanced custom models.
- Event-driven workflows that enable Needl.ai's custom RAG solutions.
- Add metrics, dashboards, and alerts (Prometheus, Grafana) to track pipeline health and catch failures early.
- Assist in system integration and deployment on cloud platforms (Azure, AWS).
- Contribute to the internal knowledge base and engineering playbooks.
- Participate in cross-functional projects with product, engineering, and customer success teams.
Qualifications
- Education: about 1 to 2 years of relevant work experience and a degree (B.Tech/M.Tech or equivalent, CSE/EE/related) from a strong institution.
Required (must have)
- Strong problem-solving ability.
- Programming in Python.
- SQL and familiarity with relational databases (Postgres/RDS).
- Good understanding of operating systems concepts (processes, memory, threading, filesystems, permissions).
- Basic understanding of cloud platforms (AWS/Azure/GCP) and deployment concepts.
- Comfortable in Linux/CLI and Git.
Nice to have
- Familiarity with coding assistants like Claude Code and Copilot.
- Experience with Elasticsearch or any other search engine.
- Kubernetes basics (pods, deployments, services).
What you'll learn
- Web scraping and crawling at scale.
- Working with production databases and search engines (PostgreSQL, Elasticsearch).
- Unstructured data extraction workflows.
- Kubernetes and scaling.
- Observability.
Common questions
Where is this role based?
Bengaluru, working hybrid: three days a week in the office, with flexible hours.
How do I apply?
Use Apply now on this page, or email your resume to careers@needl.ai with the role title in the subject line.
What does Needl.ai do?
Needl.ai is private enterprise AI for financial markets. It reads your data rooms, filings, research, email and drives, and answers the questions your investment, credit and research teams ask every day, with a citation behind every number. More on the About page and life at Needl.ai.
Other open roles
Interested? Send us your resume.
Apply now