Author Archives: MantasCode

Subreddit Relationships by Strong User Overlap

Checking out some strong subreddit relationships stemming from a chosen root subreddit. The relationships below are ordered by User Overlap Scores from subredditstats.com. An interesting way of discovering related subreddits based on user overlap.

AgainstHateSubreddits

ConsumeProduct

Interesting dichotomy of users between AgainstHateSubreddits and ConsumeProduct.


ProtectAndServe

JoeRogan top 5 of 3

Sino top 5 and 3

Coronavirus top 5 and 3

These glimpses use the User Overlap scores from https://subredditstats.com/subreddit-user-overlaps. Look up a subreddit yourself, for fun.

Top 50 most influential subreddit moderators – May 2020

Plotting number of subreddits moderated within top 1,000 most subscribed subs, vs. Subreddit Subscriber Aggregate.

Click on images below to enlarge.

The data is from http://redditlist.com/ and https://www.reddit.com/. C# was written to parse both websites and generate output formatted for a GoogleCharts Bubble Chart. This script was ran on 5/14/2020.

If you would like to parse the top 1,000 most subscribed-to subreddit mod usernames, and aggregate their frequency yourself. Feel free to download visual studio and run the code below in a simple .NET Console Application. Of course this being a hard parse it is only bound to work so long as redditlist and reddit don’t change their html structure, otherwise it can be used as a reference.

Crawl redditlist.com
Collect the middle column for the first 8 pages listing 125 each.

Crawl reddit.com
Iterate over each subreddit’s /about/moderators page and collect usernames shown below.

Regular Expression patterns used to collect information:

@"_blank(.*?)</div>"
@">(.*?)</a>"
@"listing-stat'>(.*?)</span>"
@"<span class=""user""><a href=""https://old.reddit.com/user/(.*?)/"

C# .NET Console Application

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
 
namespace RedditMegaModBubbleParse
{
    class Program
    {
        static void Main(string[] args)
        {
 
            Dictionary<string, int> dictSub_SubscriberCount = new Dictionary<string, int>();
 
            //PARSE TOP 1000 Subreddit names and subscribers//
            List<string> top500Subreddits = new List<string>();
            for (int page = 1; page < 9; page++)
            {
                WebClient wc = new WebClient();
                string htmlString = wc.DownloadString("http://redditlist.com/?page=" + page);
                MatchCollection matches = Regex.Matches(htmlString, @"_blank(.*?)</div>", RegexOptions.Singleline);
                int count = 0;
                foreach (Match match in matches)
                {
                    if (count > 125 && count <= 250)
                    {
                        Console.WriteLine(count);
                        string sHtmlChunk = match.Groups[1].Value;
                        Match mSubName = Regex.Match(sHtmlChunk, @">(.*?)</a>");
                        Match mSubscriberCount = Regex.Match(sHtmlChunk, @"listing-stat'>(.*?)</span>");
                        if (mSubName.Success && mSubscriberCount.Success)
                        {
                            string sSubName = mSubName.Groups[1].Value;
                            Console.WriteLine("Subreddit Name : " + sSubName);
                            string sSubscriberCount = mSubscriberCount.Groups[1].Value;
                            int iSubscriberCount = int.Parse(Regex.Replace(sSubscriberCount, ",", ""));
                            Console.WriteLine("Subreddit Count : " + iSubscriberCount);
                            dictSub_SubscriberCount.Add(sSubName, iSubscriberCount);
                        }
                        Console.WriteLine();
                    }
                    count += 1;
                }
            }
 
            Dictionary<string, int> dictModCount = new Dictionary<string, int>();
            Dictionary<string, int> dictModSubscriberAggregate = new Dictionary<string, int>();
 
            int count_sub = 0;
            foreach (string subreddit_name in dictSub_SubscriberCount.Keys)
            {
                try
                {
                    count_sub += 1;
                    Console.WriteLine("*** Subreddit " + count_sub + " ~~~ " + "[" + subreddit_name + "] ***");
                    WebClient wc = new WebClient();
                    string subreddit_clean = Regex.Replace(subreddit_name, " ", "");
                    string url_glue = "https://old.reddit.com/r/" + subreddit_clean + "/about/moderators";
                    Console.WriteLine(url_glue);
                    string htmlString = wc.DownloadString(url_glue);
                    MatchCollection matches = Regex.Matches(htmlString, @"<span class=""user""><a href=""https://old.reddit.com/user/(.*?)/", RegexOptions.Singleline);
                    foreach (Match match in matches)
                    {
                        Console.WriteLine(match.Groups[1].Value);
                        string modname = match.Groups[1].Value;
 
                        if (dictModCount.ContainsKey(modname))
                        {
                            int existingcount = dictModCount[modname];
                            existingcount += 1;
                            dictModCount[modname] = existingcount;
 
                            int existingSubscriberCount = dictModSubscriberAggregate[modname];
                            existingSubscriberCount += dictSub_SubscriberCount[subreddit_name];
                            dictModSubscriberAggregate[modname] = existingSubscriberCount;
 
                        }
                        else
                        {
                            dictModCount.Add(modname, 1);
                            dictModSubscriberAggregate.Add(modname, dictSub_SubscriberCount[subreddit_name]);
                        }
                    }
                }
                catch (Exception x)
                {
                    Console.WriteLine("ERROR ACCESSING SUBREDDIT  INVITE ONLY!!!!!!!");
                }
            }
 
            Console.WriteLine();
 
            string masteroutput = "";
            foreach (KeyValuePair<string, int> item in dictModCount.OrderByDescending(key => key.Value))
            {
                Console.WriteLine("['"+item.Key + "', " + item.Value+", "+ dictModSubscriberAggregate[item.Key]+", '1' , "+ dictModSubscriberAggregate[item.Key] + " ],");
                masteroutput += "['" + item.Key + "', " + item.Value + ", " + dictModSubscriberAggregate[item.Key] + ", '1' , " + dictModSubscriberAggregate[item.Key] + " ],\r\n";
            }
 
            //save to C:\Airplanes\
            StreamWriter streamWrite;
            streamWrite = File.AppendText("C:\\Airplanes\\output.txt");
            streamWrite.WriteLine(masteroutput);
            streamWrite.Close();
 
            Console.ReadLine();
 
        }
    }
}

Congrats to the May 2020 top 50 Reddit Megamods
In order of the most subreddits modded within top 1,000 most subscribed-to subreddits.

AutoModerator MAGIC_EYE_BOT BotTerminator Blank-Cheque RepostSentinel cyXie Umbresp AssistantBOT BotDefense metastasis_d awkwardtheturtle LeafSamurai Merari01 IranianGenius commonvanilla GallowBoob N8theGr8 love_the_heat ManWithoutModem greatyellowshark justcool393 Lil_SpazJoekp siouxsie_siouxv2 SuzyModQ pHorniCaiTe babar77 PowerModerator maybesaydie davidreiss666 Sunkisty yummytuber SEO_Nuke PhlogistonAster daninger4995 kjoneslol sloth_on_meth ani625 Tornado9797 sidshembekar RalphiesBoogers RepostSleuthBot Noerdy stuffed02 whyhellomichael qgyh2 Llim ModeratelyHelpfulBot EpicEngineer T_Dumbsford Kesha_Paul

126,241 Heroin Arrests Visualized, Chicago 2001 – 2020

A look into heroin related arrests spanning almost 2 decades. The data is collected by the Chicago Police Department and available at Chicago Data Portal – Public Safety section. Click on static maps below to enlarge and/or play animation.

Full city map of all incidents.

West Side of Chicago. Heroin enforcement in the west side is so dense that city blocks are fully illuminated by incident reports.

West Side of Chicago over-time animation (Geo Temporal map). Interesting migration patterns can be seen here.

South Side of Chicago (2133x2917px).

South Side of Chicago over-time animation.

Over the last two decades CPD’s enforcement of heroin has gone down.