I’m involved in a research project where my task is to collect data from reddit, and conduct social network analysis.
I’m using vosonSML to collect metadata and comments from various posts. I’m using 3,000 comments with roughly 2,500 unique users.
I’m using pullpush to find out which subreddit from a list of subreddits each user commented on the most. This requires me creating a pullpush request 3 times for each user.
Some users are taking 1-2 seconds but others are taking 1+ minutes per request. Is there anything I can do to speed this up? I’m expected to wait 15+ hours for my job to finish.
