{"id":327,"date":"2011-10-02T23:46:14","date_gmt":"2011-10-02T23:46:14","guid":{"rendered":"http:\/\/41j.com\/blog\/?p=327"},"modified":"2011-10-02T23:46:35","modified_gmt":"2011-10-02T23:46:35","slug":"find-a-random-webserver-using-libcurl","status":"publish","type":"post","link":"https:\/\/41j.com\/blog\/2011\/10\/find-a-random-webserver-using-libcurl\/","title":{"rendered":"find a random webserver using libcurl"},"content":{"rendered":"<p>Migrated from linuxjunk&#8230;<\/p>\n<p>Compile with g++ code.cpp -lcurl -o randpages. Creates random IP addresses, then attempts to grab a page from http:\/\/IP using libcurl. Reports those that reply.<\/p>\n<p>#include <iostream><br \/>\n#include <string><br \/>\n#include &#8220;stringify.h&#8221;<br \/>\n#include <curl\/curl.h><br \/>\n#include <stdlib.h><br \/>\n#include <string.h><\/p>\n<p>using namespace std;<\/p>\n<p>struct MemoryStruct {<br \/>\n  char *memory;<br \/>\n  size_t size;<br \/>\n};<\/p>\n<p>static size_t<br \/>\nWriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void *data)<br \/>\n{<br \/>\n  size_t realsize = size * nmemb;<br \/>\n  struct MemoryStruct *mem = (struct MemoryStruct *)data;<\/p>\n<p>  mem->memory = (char *) realloc(mem->memory, mem->size + realsize + 1);<br \/>\n  if (mem->memory == NULL) {<br \/>\n    \/* out of memory! *\/<br \/>\n    printf(&#8220;not enough memory (realloc returned NULL)\\n&#8221;);<br \/>\n    exit(EXIT_FAILURE);<br \/>\n  }<\/p>\n<p>  memcpy(&#038;(mem->memory[mem->size]), ptr, realsize);<br \/>\n  mem->size += realsize;<br \/>\n  mem->memory[mem->size] = 0;<\/p>\n<p>  return realsize;<br \/>\n}<\/p>\n<p>string fetch_url(string url,bool &#038;fail) {<br \/>\n CURL *curl_handle;<\/p>\n<p>  struct MemoryStruct chunk;<\/p>\n<p>  chunk.memory =(char*) malloc(1);  \/* will be grown as needed by the realloc above *\/<br \/>\n  chunk.size = 0;    \/* no data at this point *\/<\/p>\n<p>  curl_global_init(CURL_GLOBAL_ALL);<\/p>\n<p>  \/* init the curl session *\/<br \/>\n  curl_handle = curl_easy_init();<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_CONNECTTIMEOUT, 3);<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 3);<\/p>\n<p>  \/* specify URL to get *\/<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_URL, url.c_str());<\/p>\n<p>  \/* send all data to this function  *\/<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);<\/p>\n<p>  \/* we pass our &#8216;chunk&#8217; struct to the callback function *\/<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&#038;chunk);<\/p>\n<p>  \/* some servers don&#8217;t like requests that are made without a user-agent<br \/>\n     field, so we provide one *\/<br \/>\n  curl_easy_setopt(curl_handle, CURLOPT_USERAGENT, &#8220;libcurl-agent\/1.0&#8221;);<\/p>\n<p>  \/* get it! *\/<br \/>\n  CURLcode res = curl_easy_perform(curl_handle);<\/p>\n<p>  fail = false;<br \/>\n  if(res != 0) fail = true;<br \/>\n \/\/ if(doc.size() == 0) fail = true;<\/p>\n<p>  \/* cleanup curl stuff *\/<br \/>\n  curl_easy_cleanup(curl_handle);<\/p>\n<p>  \/*<br \/>\n   * Now, our chunk.memory points to a memory block that is chunk.size<br \/>\n   * bytes big and contains the remote file.<br \/>\n   *<br \/>\n   * Do something nice with it!<br \/>\n   *<br \/>\n   * You should be aware of the fact that at this point we might have an<br \/>\n   * allocated data block, and nothing has yet deallocated that data. So when<br \/>\n   * you&#8217;re done with it, you should free() it as a nice application.<br \/>\n   *\/<br \/>\n  \/\/printf(&#8220;%lu bytes retrieved\\n&#8221;, (long)chunk.size);<\/p>\n<p>  string s = chunk.memory;<br \/>\n  if(chunk.memory)<br \/>\n    free(chunk.memory);<\/p>\n<p>  \/* we&#8217;re done with libcurl, so clean it up *\/<br \/>\n  curl_global_cleanup();<\/p>\n<p>  return s;<br \/>\n}<\/p>\n<p>string rand_ip() {<\/p>\n<p>  \/\/ anything but localhost please!<br \/>\n  int top = 127;<br \/>\n  for(;top==127;) top = rand()%256;<\/p>\n<p>  string s = stringify(top) + &#8220;.&#8221; + stringify(rand()%256) + &#8220;.&#8221; + stringify(rand()%256) + &#8220;.&#8221; + stringify(rand()%256);<\/p>\n<p>  return s;<br \/>\n}<\/p>\n<p>string rand_url(bool &#038;fail) {<\/p>\n<p>  string current_address;<\/p>\n<p>  bool no_address = true;<\/p>\n<p>  string ip = rand_ip();<br \/>\n  current_address = &#8220;http:\/\/&#8221; + ip;<\/p>\n<p>  string doc = fetch_url(current_address,fail);<\/p>\n<p>  return ip;<br \/>\n}<br \/>\nint main(int argc,char **argv) {<\/p>\n<p>  size_t failed_connections = 0;<br \/>\n  size_t valid_connections  = 0;<\/p>\n<p>  if(argc < 1) cout << \"randpages <random seed>&#8221; << endl;\n\n  srand(time(NULL) + convertTo<int>(argv[1]));<\/p>\n<p>  for(;;) {<br \/>\n    bool fail = true;<br \/>\n    string url = rand_url(fail);<\/p>\n<p>    if(fail) { failed_connections++; } else { valid_connections++; }<\/p>\n<p>    if(!fail) cout << url << \" \" << valid_connections << \" \" << failed_connections << endl;\n  }\n}\n[\/sourcecode]\n<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Migrated from linuxjunk&#8230; Compile with g++ code.cpp -lcurl -o randpages. Creates random IP addresses, then attempts to grab a page from http:\/\/IP using libcurl. Reports those that reply. #include #include #include &#8220;stringify.h&#8221; #include #include #include using namespace std; struct MemoryStruct { char *memory; size_t size; }; static size_t WriteMemoryCallback(void *ptr, size_t size, size_t nmemb, void [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-327","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p1RRoU-5h","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/posts\/327","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/comments?post=327"}],"version-history":[{"count":2,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/posts\/327\/revisions"}],"predecessor-version":[{"id":329,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/posts\/327\/revisions\/329"}],"wp:attachment":[{"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/media?parent=327"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/categories?post=327"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/41j.com\/blog\/wp-json\/wp\/v2\/tags?post=327"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}