Auto Crawl Websites in SQL Database

Discussion in 'Scripts, 3rd Party Apps, and Programming' started by virtua46, Oct 14, 2014.

  1. virtua46

    virtua46 New Member

    Messages:
    12
    Likes Received:
    0
    Trophy Points:
    1
    Hi,

    I'm creating a very basic Search Engine, but i would like they crawl automatically Websites in my SQL Database (obviously with some rules, in this case only top-sites), how i can make this happening? (If you would i can share the access of the Database with you for help)

    Thanks for help! ;)
     
  2. leafypiggy

    leafypiggy Manager of Pens and Office Supplies Staff Member

    Messages:
    3,819
    Likes Received:
    163
    Trophy Points:
    63
    That would be a violation of our terms of service.
     
  3. virtua46

    virtua46 New Member

    Messages:
    12
    Likes Received:
    0
    Trophy Points:
    1
    Oh, sorry, now i know i don't try to make... (i will never known that this can be a type of violation)
     
  4. caftpx10

    caftpx10 Well-Known Member

    Messages:
    1,515
    Likes Received:
    112
    Trophy Points:
    63
    https://www.x10hosting.com/legal/terms-of-service/
    "We do not allow any type of application, bot, or script that runs continuousl or unattended." (I see a typo.)
    Crawling is basically using a bot. The rule here is very self-explanatory. Making it run automatically is an unattended and possibly a continuous task which is a no-no.
     

Share This Page