The WebSpyder School….

A Course on How to Scrape the Internet…Who is Interested??

This blog has really changed since the time I started it earlier this year. I am still working on my SEO empire and it is starting to pay off, but the majority of the traffic coming to this blog are people who are interested in learning the mechanics of “how to scrape data off the web”.

In addition to this, I get emails every single week from people who need help scraping a particular site or aren’t sure how to store data in a database and retrieve it. Some questions even go into more advanced topics like how to automate desktop applications and post the data to the web.

I am a person who believes that it is more important to “teach a person to fish” rather than give solutions on a one-on-one basis. Which got me thinking about starting up a web based school for people who want to become “Expert Web Automationist” (Yes, I did just make that up).

Actually, you can make a lot of money on freelance sites on projects that require data to be scraped off the web. The LEAST amount of money that I have made on one project was $80 when I was first starting out. Doesn’t sound like a lot of money for a project, but consider that the computer was doing all the work and it only took me 1/2 hour to setup the spider. That is $160/hr!!! And like I said that was the LEAST amount of money that I got paid to scrape data of the internet.

The school that I am envisioning will be for beginners all the way to those people who already have a lot of knowledge about programming for the web. So there will be something for everyone whether you have never written a line of code before in your life, or you are a seasoned web designer. The intent of the school will be to give you enough knowledge so you will be an “Expert” in the field of web automation, and will feel confident taking on any job for cash.

Like I said, I am only thinking about the concept of this school, but I have already put together a rough “curriculum”:

1. Web Automation Concepts
2. Web Scraping Tools – To make life easier
3. Client-Side Web Automation
   • Javascript Basics
   • IMacros Basics
   • Others
4. Server-Side Web Automation
5. Regular Expressions
6. Desktop Automation
7. Remote Objects/Yahoo PIPES/Other
8. Advanced Topics (Beating Captcha)

Each one of these topics would have many hours of videos, tutorials, practice scraping exercises, and practice exams. I will also be giving you MY CODE that I have already compiled and use on my own projects so you don’t have to create them from scratch. It is my intent to make you BETTER than I am at the art of scraping information off the internet.

If I do create the course, I will limit the amount of people who can enroll to 250 people per year. The reason being is that I don’t want to train 1000’s of people who will all flock to guru.com and drive the price of scraping down. I also don’t want 1000’s of copies of my code floating around the internet… after all it took me 100s of hours to create the code, so I only want people who are serious having it.

So what is the price??? It sure the hell won’t be $30 or $100. I haven’t narrowed it down yet, obviously because I haven’t created it yet. But if you take this course, you should be able to recoup the cost with a few freelance web scraping jobs.

So here is where I need your involvement!!

If you are interested in this course…. LEAVE A COMMENT!!!!! and let me know that you are interested. If I don’t get enough interest then I will NOT create the course. I don’t want to waste my time? Tell your friends about this if you think they are interested, tell your mother, post on forums, ect, ect.

Also email me on your thoughts on pricing, what you want to see in the course, and anything else you can think of to make the course better. If you are proficient in web scraping already and want to be involved in creating this course then let me know. If you are interested in Beta (or Alpha) testing the site let me know as well.

Notice!!! If I don’t get 100 interested people to comment to me by the 1st week of October 2008 then I will scratch the idea.

[Edit] I just noticed that some spanner is stealing my traffic, so I am extending this offer until I can get my traffic back to normal [/Edit]