in Projects, Programming, Technology

Gist Power Tip: Export Facebook Page Fans

October 13th 2011 update: This blog post is out of date. Recent Facebook changes broke the script and I’m working to fix it. Please enter your email to be notified when the new export script is available.

Your email ==>  




Businesses that embrace social media are faced with the challenge of communicating with customers on Facebook, Twitter, and other social networks. While philosophically I think this fragmentation is good, it is critical to be able extract, evaluate, and communicate with your user base wherever they may find you.

Facebook member and fan lists are not very portable. Even within Facebook, it isn’t obvious whether a Facebook Group or Facebook Page is better for communicating with customers. Gist is an emerging as a social Swiss Army knife (and social CRM) that can help solve this problem.

One thing that has always annoyed me about Facebook Pages is that there is no way to export fans. I’ve discovered workable hacks for downloading the member list from both groups and pages.

For Facebook groups, you can use the handy Export group members to CSV application. It uses the Facebook API to export members in to a handy Comma Separated Value file, which you can in turn import into Outlook or Gist.

Export Facebook group members

Using the EGMCSV app, select your group and the attributes “Facebook ID”, “Full name”, and “Picture URL”. Once you have a CSV file, edit the column names to those below.

  • facebook_id
  • Name
  • facebook_pic_big

These are undocumented CSV columns that Gist supports. They are required to track that the contact came from Facebook. Gist will automatically download their photo.

Import CSV into Gist

In Gist, click the “Account” link, then “Other/CSV” under “Connect More Accounts”. Once Gist has processed your file, contacts will start to appear.

Gist contact import

Getting fans from a Facebook page is not yet supported by the Facebook API. Luckily, the Facebook Web interface uses a simple AJAX/JSON call to supply the data when you view the page.

Gist Fans

My strategy to set this data free was to sniff the network traffic with the Wireshark tool, then replay the HTTP calls with a ruby script. The script below will iterate over the page’s fans, save the pages as JSON in plain text files, then load the text files and convert them to CSV files in the format we used above for groups. Note that if you run this you will need to substitute the value of your cookies and the form values in the HTTP post body. This insures you are authenticated as yourself when you connect to Facebook.

FacebookFanExport

Here is the text-friendly version of the Facebook Page Fan export script. I didn’t try this with pages for which I am not the administrator. I don’t see any reason why this wouldn’t work for your competitors’ fans.

The net result is a consolidated list of fans (prospects and leads) in Gist.

Gist People List

Gist will help de-dup contacts whether they originated from Facebook or Twitter. You could for example, use Gist to identify customers from your Web site that are on Twitter, and generate a special Twitter focused mailing to them. Gist has a ways to go to make this easier (bulk contact tagging by import), and also does not yet automate the communication side of the process (send a message to a person regardless of what network they are on).

Write a Comment

Comment

89 Comments

  1. Hi Adam,

    Like the article, interested to know if this can be applied to sites such as “linkedin”. I use this a lot as it’s more business person focused.

    If it can that would be a great help in list building.

    Regards

    Andy

  2. Adam, great article. We are seeking the industry for the right kind of people to develop Gist in the Software side and this really does a great job explaining how it works. Thank you, -Blake Berryman (Technical Recruiter-Verticalmove)

  3. interesting article, Adam. Whenever I do social media, I’ve always created fan pages instead of group pages. What really is the difference between the two? Thanks for the info, I know a couple people who’ll find this extremely useful.

  4. I have been searching for such a thing since a week. I have facebook fans and i would like to share with them my views on our personal pages, now when i have seen this page i got to know exactly know what exactly it is.

  5. Hi there,

    Any chance of an even more dumbed-down, step-by-step tutorial on how to get the fans list out of Facebook? What exactly needs to be done with the script?

    Many thanks.

  6. Wow – that is intense. Good show.

    Here is a php script that uses cURL that does it too in case anyone does not have access to Ruby. I added/modified Christian’s code (see below) that was a status updater to make it a Page Fan list extracter. I noticed that Adam made the header a Mac – if you run this on a mac Leopard it seems to have cURL as part of it’s php. I am not sure about Tiger.

    <?PHP
    /*******************************
    * Facebook Status Updater
    * Christian Flickinger
    * http://nexdot.net/blog
    * April 20, 2007
    *******************************/

    $login_email = 'YOUR_USER_NAME';
    $login_pass = 'YOUR_PASSWORD';

    fb_get_fans($login_email, $login_pass);

    function fb_get_fans($login_email, $login_pass){

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, 'https://login.facebook.com/login.php?login_attempt=1&#039;);
    curl_setopt($ch, CURLOPT_POSTFIELDS,'charset_test=%E2%82%AC%2C%C2%B4%2C%E2%82%AC%2C%C2%B4%2C%E6%B0%B4%2C%D0%94%2C%D0%84&locale=en_US&email='.urlencode($login_email).'&pass='.urlencode($login_pass).'&pass_placeholder=&charset_test=%E2%82%AC%2C%C2%B4%2C%E2%82%AC%2C%C2%B4%2C%E6%B0%B4%2C%D0%94%2C%D0%84');
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_COOKIEJAR, str_replace('\','/',dirname(__FILE__)).'/fb_cookies.txt');
    curl_setopt($ch, CURLOPT_COOKIEFILE, str_replace('\','/',dirname(__FILE__)).'/fb_cookies.txt');
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6 (.NET CLR 3.5.30729)");
    $result = curl_exec($ch);

    $page = '';
    $urls = '';
    $stop = false;
    $i=0;
    while ($stop==false && $i0?(‘&start=’.$i):”);
    curl_setopt($ch, CURLOPT_URL, $url);
    $urls .= $url.”n”;
    //echo $url;
    $thispage = curl_exec($ch);
    $i = $i +10;

    // 101-20 of 20 results
    $matches = Array();
    if (preg_match(‘/(d+)-(d+) of (d+) results/’, $thispage, $matches)) {
    //print_r($matches[0]);
    if ($matches[1]>$matches[3] && $matches[2]==$matches[3]) {
    $stop=true;
    }
    }
    $page .=$thispage;
    }
    //echo $page;

    //echo “nnnnnENDnnnnnn”;
    $matches = Array();
    if (preg_match_all(‘/Name:.* UIFullListing_FirstInfoValue”>.* onclick=””>([^>]*)

  7. // And here is the rest – it seems to have cut if off. Look for the overlap

    //echo “nnnnnENDnnnnnn”;
    $matches = Array();
    if (preg_match_all(‘/Name:.* UIFullListing_FirstInfoValue”>.* onclick=””>([^>]*)

  8. OK – lets try posting this once more – the easy way…

    Wow – that is intense. Good show.

    Here is a php script that uses cURL that does it too in case anyone does not have access to Ruby. I added/modified Christian’s code (see below) that was a status updater to make it a Page Fan list extracter. I noticed that Adam made the header a Mac – if you run this on a mac Leopard it seems to have cURL as part of it’s php. I am not sure about Tiger.

    See See my script

  9. I’m trying to convert this script in php.
    I’m receiveing:
    [error] => 1357004 [errorSummary] => Authentication Failure [errorDescription] => You are not permitted to do that.

    Is this now happening also with Ruby version?

  10. @ tempus:
    same question here…

    downloaded whireshark, got a Gist account now. but don’t really have an idea where to put the script in…

    thanks for help!

  11. It looks like a very useful script. Could you tell a non-techie person how and where to save it to be able to run it.
    Thanks a lot
    JP

  12. Hi Adam,

    Thanks for sharing such a great tool. If you have time I would love to hear some thoughts from you on this error that I get when I run it (I have inserted all the correct header info etc. (as far as I know):

    facebook.rb:35: undefined method `[]’ for false:FalseClass (NoMethodError)
    from facebook.rb:31:in `times’
    from facebook.rb:31

    Noob question probably – sorry!

    Joe

  13. I really need a copy of this script. I tried Adam’s python script and it doesn’t work properly. Your link to the text version of the script seems to be down.

    Any way you can repost it? I have a page with 15,000+ fans and need to export them for a contest draw we are doing.

    Thanks

  14. I give up. I cannot get this thing to work. I have ruby running on windows 7 and I replaced the cookie field using wireshark and replaced the user-agent and the X-Svn-Rev info. Could use a little more detail on the steps to make this work.

  15. i think i did every thing right but am getting the following error
    Got Page 0
    Got Page 1
    Got Page 2
    Got Page 3
    Got Page 4
    facebook_id,Name,City-state,facebook_pic_big
    facebook.rb:29:in `block in ‘: undefined
    (NoMethodError)
    from facebook.rb:25:in `times’
    from facebook.rb:25:in `’

    the .js file created have text that shows that the login was not successfully

    any idea why?

  16. i don’t believe Adam intends to have others use this script. If he did there would be a worked out example. This is more of a proof of concept at this point.

  17. Hello Adam,

    Very cool post! works great! Only problem is it seems to stop at 10 000 fans, then the script returns the same group of people.. (this is on a fan page where I am not admin), any ideas ?

    thanks!

    Joe

  18. I have a facebook fan page with over 10,000 fans!
    I amlooking for a script that will allow me FIND some “people who like this page” fast, to be able to remove them.

    I can click on “see all” then a window opens, but, since I have 10,000, it is almost impossible to find them. So the script would help to FIND a specific fan FASTER….any idea? please write to almtz65@gmail.com

  19. I rewrote it in PHP for people who don’t have ruby… thought it might help someone else.


    <?php
    set_time_limit(0);
    // Hamish - 27 july 2010
    // use charles http proxy http://www.charlesproxy.com/ to get the cookie string and node_id, post_form_id, fb_dtsg for the $postFields string
    // you need to have curl enabled on your web server

    $fnRaw = "posh_fb_output.txt";
    $fnCsv = "posh_fb_output.csv";

    // set settings in these 4 lines from results of charles when getting the 2nd page of "Get All Fans" in FB ( you need to be admin of fan page to do this )
    $cookie="datr=123123123123123; lo=123123123; lxs=1; locale=en_US; x-referer=http%3A%2F%2Fwww.facebook.com%2Fname%3Fref%3Dts%23%2Fhome.php; cur_max_lag=2; lsd=2wrNo; c_user=112312312313; lxe=name%40name.com; sct=123123123; xs=123123123; presence=D12312321323213";
    $node_id= "12312313123";
    $post_form_id = "123123123";
    $fb_dtsg = "QT-pZ";
    // end settings

    $url= "http://www.facebook.com/ajax/social_graph/fetch.php";
    $ch = curl_init();
    $stop = false;
    $page=0;
    while(!$stop)
    {
    $postFields = "page=".$page."&edge_type=fan&limit=100&class=FanManager&node_id=".$node_id."&post_form_id=".$post_form_id."&fb_dtsg=".$fb_dtsg."&post_form_id_source=AsyncRequest&__a=1";

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt ($ch, CURLOPT_POST, 1);
    curl_setopt ($ch, CURLOPT_POSTFIELDS, $postFields);
    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; MDDR; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)");
    curl_setopt($ch, CURLOPT_COOKIE, $cookie);
    $r = curl_exec ($ch);
    WriteOut($fnRaw,$r);

    echo "got page: ".$page."";
    ob_flush();
    if(preg_match ('/"user_info":null/' ,$r,$matches) > 0) $stop = true;
    $page++;
    }
    curl_close ($ch);

    // decode file
    WriteOut($fnCsv,"ID,PROFILE HREF,TITLE, SUBTITLE, PICTURE URL");
    $fp = fopen($fnRaw,"r");
    while (!feof($fp))
    {
    $line = fgets($fp, filesize($fnRaw));
    $line = str_replace("for (;;);","",$line);
    $j = json_decode($line,true);
    if($j["payload"]["user_info"])
    {
    foreach($j["payload"]["user_info"] as $p)
    {
    WriteOut($fnCsv,$p["id"].",".$p["href"].",".$p["title"].",".$p["subtitle"].",".$p["pic"]);
    }
    }
    }
    fclose($fp);
    echo "complete - have a great day.";

    function WriteOut ($fn,$theData) {
    $fp = fopen($fn,"a");
    fwrite($fp,$theData."rn");
    fclose($fp);
    }
    ?>

  20. Hi Adam, I left in some of my FB settings in the script by mistake, can you remove them before publishing.
    Cheers
    Hamish

  21. Hi Hamish,

    I am using your script. But I cannot find the node_id in the proxy tool, where can I find the node_id. I would appreciate any help from you.

    Thanks.

  22. @hamish

    very nice solution! tried to get it work with exporting all my friends as well, after examining with CHARLES i found out that only params changed were: $node_id and $cookie … so i copied those into your script, but it is not working somehow… keeps running forever. can you check this please and tell if it works for you?

  23. I get the same error message as Joe:
    fb.rb:35: undefined method `[]’ for false:FalseClass (NoMethodError)
    from fb.rb:31:in `times’
    from fb.rb:31

    version of ruby? some classes I’m missing?

  24. I was able to use Hamish’s code and it worked fine. Just make sure to copy the full cookie string from Charles proxy and replace the one in Hamish’s code with it ($cookie).

    If you see that the php once run keeps running (got page: x over and over), then it’s prob not working and you should stop it.

    Check the folder the file is running in via FTP – if fboutput.txt exists without fboutput.csv – it means it didn’t work. THe way the script works, the txt is created first, then processed into the csv.

  25. Adam – Thanks for sharing the script. I was able to export our fans easily. Is there another variation to enable me to export the list of pages we ‘like’? – Thanks again, –D

  26. I found your site, desperate for some help! This above “Facebook group” has over 8,000 members and is a group dedicated to Ryan Diviney, a student left in a coma after an assault. With the new archiving and migration Facebook effort, the father of this Facebook group does not know how he will save those contacts (FB will not migrate them) or at least export them so he can invite them to the new group. Please please can you help us! Not the best technological group, but I can get people to help. Thank you so much!

  27. Hi Adam, we are very interested in this script you have made, we are looking to Export Numerous Emails from Australian Facebook Fan Pages

    $1000.00 AUD per 100,000 Emails Exported. up to a maximum of 500,000 Emails. Neg* AT any breakdown that works for you
    $100.00 for 10K, etc.

    Please let me know if you are interested or can direct me to somebody who has the time to do this.

  28. Thanks. I don’t find anywhere I can see the list of a fan page (where I am not an administrator). I had see that in the past!

    James of OZ, there isn’t any emails in the list exported. (I know there are people supplying them)

  29. can someone confirm before i go through all these steps if this exports the email addresses for the fans of the page? or only the names and facebook ids? i need email addresses for my fans.

  30. Hi Adam,

    I have been trying to use your facebook code, with no avail to export fans form a page I am admin of. This is the script that I am using. Please help 🙂

    var uri = ‘/ajax/social_graph/fetch.php?__a=1’;
    var lastResponse = null;
    var usersPerPage = 100;
    var totalUsersDownloaded = 0;
    var throttle = 3000; // how long to wait between pages
    var startPage = 0;
    var endPage = 10; // change this if you have more than 1,000 fans

    // Find the social graph node ID (page profile ID) by peeking at meta tags
    var getNodeId = function() {
    var metaTags = document.getElementsByTagName(‘meta’);
    for (i in metaTags) {
    var tag = metaTags[i];
    if (tag.content && tag.content.match(/_([0-9]+)_/)) {
    return tag.content.match(/_([0-9]+)_/)[1];
    }
    }
    return null;
    }

    // Process the AJAX call response and dump the user data to the console
    var OnResponse = function(e) {
    console.log(‘— Page: ‘ + e.payload.page);
    lastResponse = e; // for debugging

    for (userId in e.payload.user_info) {
    var userData = e.payload.user_info[userId];
    totalUsersDownloaded++;
    console.log(userId + ‘,’ + userData.title + ‘,’ + userData.subtitle + ‘,’ + userData.pic)
    }

    console.log(‘Downloaded: ‘ + totalUsersDownloaded + ‘ of ‘ + e.payload.count)
    if (e.payload.page <= endPage && totalUsersDownloaded < e.payload.count) {
    setTimeout(function() { downloadUsers(e.payload.page + 1); }, 3000);
    }
    }

    // Make an AJAX call for the data using FB's AJAX library
    var downloadUsers = function(page) {
    var nodeId = profileid;

    if (!nodeId) {
    alert('Sorry couldn't find profile ID');
    return;
    }

    var data = {
    edge_type: 'fan',
    page: page,
    limit: usersPerPage,
    node_id: nodeId,
    'class': 'FanManager',
    post_form_id: document.getElementById('post_form_id').value,
    fb_dtsg: document.getElementsByName('fb_dtsg')[0].value,
    lsd: null,
    post_form_id_source: 'AsyncRequest'
    }

    var req = new AsyncRequest()
    .setURI(new URI(uri))
    .setMethod('POST')
    .setData(data)
    .setHandler(OnResponse);

    result = req.send();
    }

    downloadUsers(startPage);

  31. Hi Adam,

    I would like to export my fans from one Facebook page to another. I can’t request a merge because they have slightly different names but they are the same musician (me). Is there a way to do this?

  32. I tried script from git in chrome and firefox with profile_id (nodeId) hardcoded (and dinamic from meta) and returns a 404 status after post, it seems they changed something in their ajax requests or is “me”:D.

    • Yes, the inner workings of Facebook changed. I’ve got a fix for those who previously purchased the premium script. It scrapes the data from DOM nodes instead of using Facebook’s AsyncRequest libraries.

  33. I find the new ajax route but it needs a little reverse engineering of the scripts. I hope it will be an update if i can’t/can hack this:D.

  34. Adam, I was a big fan of your old AJAX script. Have you figured out a new way to do this? I would purchase a “premium” script as well.

  35. You mentioned a work around for buyers of the premium script. Is this still for sale? If so I’m interested. Shoot me an email.

  36. When I copied the code and pasted it on javascript console (Chrome) it returns: SyntaxError: Unexpected token ILLEGAL

    Please help.

  37. I sure could use a working script too! Just trying to get a list of likes off my business page so I can select a random person from the list of likes. Really just need names in the simplest sense. Any luck on an updated script? Or a premium version? Thanks much!

  38. I’m interested in using this script too. please email me if you’ve gotten this to work. Happy to get a paid version as well. Thanks.

  39. Hi Adam,
    I have face book page .That page is liked by more face book user.Is there any way to get face book user email id and profile id.Please help me in this problem if you know

  40. Adam, as with everyone, I am also interested in getting the emails of the “likers” of the pages I admin. Do advise if you are still offering a premium version for our purpose. Thanks!

  41. Adam, interested in getting the email ids of about 0.3 Mn members on our fan page but not familiar with scripting. Can you help us extract these ids without putting the page at risk from FB? How much would that cost?

  42. Hey,

    It seems the new url is “https://www.facebook.com/ajax/browser/list/page_fans/?page_id=[THE_PAGE_ID]&edge=public_profile%3Afbpage_to_user&start=[OFFSET]&__user=[YOUR_USER_ID]&__a=1” , but there is a limit of 500 users returned … bummer.

    • Sherryl – Thanks for checking in. I haven’t researched it lately, and haven’t heard anything from Facebook about it. It has been quite a while so I will look again when I have the chance!

  43. hello – I was trying to order a script from you on Fiverr –if that’s you. They sent me to this link. Can you please contact me?

Webmentions

  • How to Transfer Your Facebook Friends to Twitter & Other Networks [Bookmarklet] July 16, 2021

    […] friends– not your Facebook page’s friends. I’m hoping that Adam Loving will update his Facebook Page Hack for this! (Any update on […]

  • Killing Facebook Personal Pages For Your Nonprofit » Social Needia July 16, 2021

    […] is an old script that used to scrape your likes and allow you to export to csv and Excel (here).  Unfortunately, Facebook has seemed to have caught up to this script and doesn’t love that […]

  • How to Migrate your Facebook Friends to Twitter and Google+ | Ian Anderson Gray July 16, 2021

    […] your Face­book pages. There have been many hacks that have tried to achieve this (not­ably this Javas­cript hack by Adam Lov­ing — @adamloving) but many either don’t work or have been removed, pre­sum­ably […]

  • Beth Granter – Interesting data projects and articles July 16, 2021

    […] Gist Power Tip: Export Facebook Page Fans – if out of date, check comments […]

  • Is it possible to track the source of fans of a Facebook fanpage? - Quora July 16, 2021

    […] try and I don't know if they can also show what you are asking for, but maybe it helps a bit.http://adamloving.com/internet-p…Insert a dynamic date hereCannot add comment at this […]

  • Gagner des bijoux Molusk sur Facebook ! | Molusk : Création de bijoux fantaisie en plastique contemporain July 16, 2021

    […] n’ai pas trouvé de méthode simple pour faire un tirage au sort parmi nos fans Facebook, ou pour exporter cette liste en format .CSV. […]