🚀 B2B Lead Scraper

Automated B2B Lead Generation Tool

Find qualified leads, extract emails, scrape company data, and push to your CRM

Features • Installation • Quick Start • CLI Commands • CRM Integration • Contributing

---

✨ Features

🔍 Lead Discovery

• Google Search Integration - Find companies by industry, location, and keywords
• LinkedIn Scraping - Extract company profiles and information
• Website Scraping - Deep scrape websites for contact information

📧 Email Extraction

• Pattern-Based Discovery - Generate potential emails from name + domain
• Confidence Scoring - Prioritize high-confidence email addresses
• Multi-Level Validation - Syntax, DNS MX, and optional SMTP verification

🏢 Company Data

• Comprehensive Profiles - Name, industry, size, location, social links
• Technology Detection - Identify tech stack from website analysis
• Structured Data Extraction - JSON-LD and meta tag parsing

📊 Export Options

• CSV Export - Excel-compatible with customizable columns
• Excel Export - Formatted spreadsheets with multiple sheets
• JSON Export - API-ready format with NDJSON support

📝 Email Templates

• Cold Outreach - Personalized first-touch templates
• Follow-ups - Multi-stage follow-up sequences
• Variable Substitution - Dynamic personalization with {{variables}}

🔗 CRM Integrations

• Salesforce - Create Leads, Contacts, and Accounts
• HubSpot - Create Contacts and Companies with associations
• Extensible - Easy to add new CRM adapters

---

📦 Installation

Prerequisites

• Python 3.9 or higher
• pip package manager

Quick Install

# Clone the repository
git clone https://github.com/yourusername/sales-lead-scraper-tool.git
cd sales-lead-scraper-tool

# Create virtual environment
python -m venv venv

# Activate virtual environment
# Windows:
venv\Scripts\activate
# macOS/Linux:
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Install as editable package (optional)
pip install -e .

Configuration

1. Copy the example configuration:

cp .env.example .env
cp config/config.example.yaml config/config.yaml

2. Edit .env with your CRM credentials:

# Salesforce
SALESFORCE_USERNAME=your_username
SALESFORCE_PASSWORD=your_password
SALESFORCE_SECURITY_TOKEN=your_token

# HubSpot
HUBSPOT_ACCESS_TOKEN=your_token

---

🚀 Quick Start

Search for Leads

# Search for software companies
python -m cli.main search "software companies"

# Search with filters
python -m cli.main search "marketing agencies" -l "New York" -n 50

# Export results to Excel
python -m cli.main search "saas startups" -f excel -o leads.xlsx

Extract Emails from Website

# Basic extraction
python -m cli.main extract https://example.com

# Deep scrape with email validation
python -m cli.main extract https://example.com --deep --validate

Generate Email Patterns

# Generate potential emails from name
python -m cli.main generate John Doe acme.com

# With DNS validation
python -m cli.main generate John Doe acme.com --validate

Push to CRM

# Push leads to Salesforce
python -m cli.main push leads.csv --crm salesforce --type lead

# Dry run to preview
python -m cli.main push leads.csv --crm hubspot --dry-run

---

💻 CLI Commands

Command	Description	Example
search	Search for leads by keyword	search "tech startups" -l "SF"
extract	Extract data from website	extract example.com --deep
generate	Generate email patterns	generate John Doe acme.com
export	Convert between formats	export leads.json -f csv
push	Push leads to CRM	push leads.csv --crm hubspot
template	Generate email from template	template cold_outreach -v name=John
validate-config	Check configuration status	validate-config

Global Options

python -m cli.main --help           # Show all commands
python -m cli.main search --help    # Show command options
python -m cli.main --version        # Show version

---

📋 Python API Usage

Search for Companies

from src.scrapers import GoogleScraper

with GoogleScraper() as scraper:
    results = scraper.search_companies(
        industry="technology",
        location="San Francisco",
        max_results=50
    )
    
for company in results:
    print(f"{company['title']} - {company['url']}")

Extract Emails

from src.scrapers import WebsiteScraper
from src.extractors import EmailExtractor
from src.validators import EmailValidator, ValidationLevel

# Scrape website
with WebsiteScraper() as scraper:
    data = scraper.scrape_website("https://example.com", deep_scrape=True)

# Extract and validate emails
extractor = EmailExtractor()
validator = EmailValidator()

for email in data['emails']:
    result = validator.validate(email, ValidationLevel.DNS)
    if result.is_valid:
        print(f"✓ {email}")

Generate Email Patterns

from src.extractors import EmailExtractor

extractor = EmailExtractor()
patterns = extractor.generate_patterns("John", "Doe", "acme.com")

for p in patterns:
    print(f"{p.email} (confidence: {p.confidence:.0%})")

Export to CSV

from src.exporters import CSVExporter, ExcelExporter

# Export to CSV
csv_exporter = CSVExporter()
filepath = csv_exporter.export(leads, "my_leads.csv")

# Export to Excel with formatting
excel_exporter = ExcelExporter()
filepath = excel_exporter.export_with_summary(leads)

Push to CRM

from src.integrations import SalesforceIntegration, HubSpotIntegration
from src.integrations.base_crm import CRMRecord, RecordType

# Create record
record = CRMRecord(
    record_type=RecordType.LEAD,
    first_name="John",
    last_name="Doe",
    email="[email protected]",
    company_name="Acme Inc"
)

# Push to Salesforce
with SalesforceIntegration() as sf:
    result = sf.create_lead(record)
    print(f"Created lead: {result.record_id}")

# Push to HubSpot
with HubSpotIntegration() as hs:
    result = hs.create_contact(record)
    print(f"Created contact: {result.record_id}")

Use Email Templates

from src.templates import TemplateEngine

engine = TemplateEngine()

# List available templates
print(engine.list_templates())

# Render template
email = engine.render_template("cold_outreach", {
    "first_name": "John",
    "company_name": "Acme Inc",
    "sender_name": "Jane Smith",
    "sender_company": "Our Company"
})

print(email)

---

🔗 CRM Integration

Salesforce Setup

1. Get your Security Token from Salesforce Settings
2. Add credentials to .env:

SALESFORCE_USERNAME=[email protected]
SALESFORCE_PASSWORD=your_password
SALESFORCE_SECURITY_TOKEN=your_token
SALESFORCE_DOMAIN=login  # Use "test" for sandbox

HubSpot Setup

1. Create a Private App in HubSpot Settings
2. Grant scopes: crm.objects.contacts.write, crm.objects.companies.write
3. Add access token to .env:

HUBSPOT_ACCESS_TOKEN=pat-na1-xxxxxxxx

---

📁 Project Structure

sales-lead-scraper-tool/
├── cli/                    # Command-line interface
│   └── main.py            # CLI entry point
├── config/                 # Configuration
│   ├── settings.py        # Settings management
│   └── config.example.yaml
├── src/
│   ├── scrapers/          # Web scrapers
│   │   ├── google_scraper.py
│   │   ├── linkedin_scraper.py
│   │   └── website_scraper.py
│   ├── extractors/        # Data extractors
│   │   ├── email_extractor.py
│   │   └── company_extractor.py
│   ├── validators/        # Data validators
│   │   └── email_validator.py
│   ├── exporters/         # Export formats
│   │   ├── csv_exporter.py
│   │   ├── excel_exporter.py
│   │   └── json_exporter.py
│   ├── templates/         # Email templates
│   │   ├── template_engine.py
│   │   └── email_templates/
│   ├── integrations/      # CRM integrations
│   │   ├── salesforce.py
│   │   └── hubspot.py
│   └── utils/             # Utilities
│       ├── logger.py
│       ├── rate_limiter.py
│       └── proxy_manager.py
├── tests/                  # Test suite
├── output/                 # Default export directory
├── requirements.txt
└── README.md

---

🧪 Running Tests

# Run all tests
python -m pytest tests/ -v

# Run with coverage
python -m pytest tests/ -v --cov=src

# Run specific test file
python -m pytest tests/test_validators.py -v

---

⚙️ Configuration Options

config.yaml

scraper:
  rate_limit_delay: 2.0      # Seconds between requests
  max_retries: 3             # Retry failed requests
  timeout: 30                # Request timeout
  use_proxy: false           # Enable proxy rotation
  respect_robots_txt: true   # Honor robots.txt

email:
  validate_dns: true         # Check MX records
  validate_smtp: false       # SMTP verification (slow)
  common_patterns:           # Email pattern templates
    - "{first}.{last}"
    - "{first}{last}"
    - "{f}{last}"

export:
  output_directory: "./output"
  csv_delimiter: ","
  include_timestamp: true
  default_format: "csv"

---

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

⚠️ Disclaimer

This tool is intended for legitimate B2B lead generation purposes. Always:

• Respect website terms of service and robots.txt
• Comply with data protection regulations (GDPR, CCPA)
• Use rate limiting to avoid overloading servers
• Only contact businesses who may genuinely benefit from your services

---

📞 Support

• 📫 Open an issue for bug reports or feature requests
• 💬 Start a discussion for questions or ideas

---

Built with ❤️ for Sales & Marketing Teams