Data gathering code for developers

Learn how to create a customizable Python web scraping script to automatically gather and store specific data from any website.
Back to prompts
Category
Coding
Stats
46 views
1 copy
0 likes

Prompt

Open in ChatGPT
Copy
Hey ChatGPT, act as a software engineer. Write a Python script that scrapes data from
{{website}}
. The website is a
{{type}}
and I want to gather the
{{data}}
. The program should run on a
{{os}}
and be designed to run
{{frequency}}
. If any errors are encountered, the script should
{{error}}
. The scraped data should be saved in a
{{format}}
and stored at
{{storage}}
.
Website
Type
Data
Os
Frequency
Error
Format
Storage

This prompt helps you generate a custom Python web scraping script tailored to your specific data collection needs. By providing details about the target website and your requirements, you'll receive ready-to-use code for automated data gathering.

How to Use

Simply provide information for each of the required variables in the prompt. Be as specific as possible about what data you need to collect and how you want to handle it. The generated script will be adapted to your specifications for the target website.

For best results:

  • Be specific about the exact data points you need
  • Consider ethical scraping practices and website terms of service
  • Provide clear information about how often you plan to run the script

Variables Explained

  • {{website}}: The URL of the website you want to scrape (e.g., "https://example.com/products")
  • {{type}}: Type of website (e.g., "e-commerce site," "news portal," "forum," "blog")
  • {{data}}: Specific information you want to extract (e.g., "product prices and names," "article headlines and dates," "user comments")
  • {{os}}: Operating system where the script will run (e.g., "Windows 10," "macOS," "Ubuntu Linux")
  • {{frequency}}: How often the script should run (e.g., "daily," "hourly," "every 15 minutes," "once")
  • {{error}}: Error handling behavior (e.g., "log errors to a file," "send email notifications," "retry three times")
  • {{format}}: Output data format (e.g., "CSV," "JSON," "Excel," "SQL database")
  • {{storage}}: Where to store the collected data (e.g., "local directory," "S3 bucket," "Google Drive," "MySQL database")

Tips for Better Results

  • Specify any authentication requirements if the website requires login
  • Consider rate limiting to avoid being blocked by the website
  • Be aware that some websites have anti-scraping measures in place
  • For complex websites, provide details about the site structure if you know them
  • Remember that web scraping should be done responsibly and legally
Want to 10x your AI skills?

Get a free account and learn to code + market your apps using AI (with or without vibes!).

Nah, maybe later