Topstone Software Consulting

The Architecture of nderground (www.nderground.net)
a social network built for privacy

The design philosophy for nderground

The honeymoon with social media has ended. There’s nothing in it for people who have anything to lose.
 Social Media is Dead to Donald Glover, Action Bronson and Iggy Azalea by Paul Cantor, Medium, Feb 25, 2015

Humans are social creatures. We crave connection, recognition and affection from other people. Social networks like Facebook have been designed to monetize our desire for human connection. In fact, social networks are deigned to turn our desire for connection into a compulsion.

The currency that pays for social network access is our personal information, which becomes the input to sophisticated software analysis that processes this information as fodder for advertisers of all types.

The image of life that is presented on social networks is often a synthetic false image of perfect lives, relationships and gourmet meals. Academic studies suggest that people feel worse looking at social network posts as users measure their actual lives against the carefully curated lives they see on social networks.

nderground is a social network designed for privacy. Nothing posted on nderground is ever visible on the internet. Writing and photographs posted on nderground can only be seen by a select group of friends and family. nderground is designed to be a more authentic environment.

As of right now, my entire Facebook feed is made up of pictures, comments, life events and more of people I’ve never heard of before – almost exclusively people I’ve never heard of or seen before. It seems to be an experiment where my entire feed is populated by posts by people who I do not know but ones that my Facebook friends have liked or commented on.

It sucks. But more than that it is eye-opening and a little jarring to me just how disconcerting and unpleasant it is. Sartre said that hell is other people; actually it turns out to be the life events of people you don’t know.
Josh Marshall, Talking Points Memo, July 1, 2015

A common (and unnerving) experience on Facebook takes place when you post a comment directed at a friend, only to have it packed up by people you don't know in your friend's social circle. In the worst case, a careless comment can be picked up by strangers, go viral, and sow chaos in your life.

On nderground posts and photos are only seen my your immediate social circle. We call this social circle a karass, a term from Kurt Vonnegut's novel Cat's Cradle. Unless they are also your friends, the friends of your friends will never see anything you post on nderground.

The Karass and Privacy on nderground

On nderound a Karass is the group of people who are part of your life. Your friends, your family or whoever you want to invite into your world on nderground. The content that you post on nderground can only be seen by your Karass when they are logged into nderground.

In the diagram below Bob, Alice and Ted are in the same Karass. They can all see material that each of them posts on nderground.

Sarah is in Bob's Karass, but not in Alice's Karass and Jane is in Alice's Karass, but not in Bob's. This means that Bob and Sarah can each see material that each of them posts, but Alice cannot see any of the material that Sarah posts. Similarly, Jane is in Alice's Karass and can see what Alice posts, but Jane can't see anything posted by Bob or Ted.

nderground Engineering Objectives

Security and Privacy

The core of the nderground social network is privacy. In order to protect privacy, nderground must be a "hard target" web application, using the latest web security features.

Scalability

nderground is designed to support hundreds of thousands of users. As users make demands on nderground, resources should smoothly be added to handle the load.

Cost

nderground is a bootstrap startup. There's no venture capital money to burn. The founders of nderground pay all of the bills, so a design that keeps costs as low as possible is important.

The nderground Board and Photo Galleries


The nderground board for one of the test accounts


The nderground photo galleries for the authors nderground account

The two foundational features of nderground are the Boards and photo galleries belonging to nderground users. The Board allows an nderground user to post text and photos. Other members of the user's Karass see and can comment on board posts. An nderground user may also visit the Board of a member of their Karass and post or comment.

Some social network users like to interact with their community by posting photos, perhaps with brief captions. nderground supports named photo galleries that allow nderground users to post photos that they want to share with their Karass.

nderground on Amazon Web Services

The opportunity to leverage Amazon Web Services has allowed nderground to be ready for users faster, at lower cost. All of the services utilized by nderground are "zero maintenance", allowing time to be focused on developing nderground, not maintaining web and database servers.

The diagram below outlines the AWS services utilized by nderground.

  • Route 53 DNS Resolution

    One of the first steps in satisfying a Web request is Domain Name Service (DNS) resolution, which maps a domain name, like nderground.net, to an actual IP address. The Route 53 service makes it easy to set up DNS resolution for domains that are hosted on AWS.

  • Elastic Beanstalk

    Elastic Beanstalk is a load balancing web server environment. One or more web servers can be configured to run a web application. As demand increases, Elastic Beanstalk will add additional web servers to handle the load. When demand decreases again, web servers that are allocated to handle demand will be scaled back.

    nderground is a Java Virtual Machine (JVM) web application. To run nderground, Elastic Beanstalk is configured to run the Apache Tomcat JVM application web server. AWS provides the latest versions of Apache Tomcat and the JVM environment. Upgrading to a new version takes a click of a button.

  • Elastic Compute Cloud (EC2) Linux Instances

    When the Elastic Beanstalk environment is configured for a web application, one or more EC2 instances are selected. In the case of a Tomcat application server these instances will run Amazon's version of Linux. To save money Amazon "reserved instances" can be purchased. These instances will automatically be used in the Elastic Beanstalk configuration.

    As with any EC2 Linux instance, the user can use ssh to login and examine the web logs, which can be useful when tracking down execution related problems.

  • RDS/Postgres

    Amazon's Relational Database Service (RDS) provides "managed" (e.g., zero maintenance) database instances. nderground uses the RDS/Postgres database to store critial information where transactional update is important. This includes the nderground encrypted login information and Karass membership information.

    The Postgres database is lightly utilized since it includes only a relatively small amount of critical information. Although the database is lightly used, an RDS server that runs 24/7 must be allocated.

    In November 2017 Amazon announced the Serverless Aurora service. Aurora is Amazon's scalable relational database. Aurora supports both MySQL and Postgres versions. The MySQL version of Aurora became available in early 2018. The Postgres version is scheduled to be available in the second half of 2018.

    Severless Aurora is a very attractive service for a web applications like nderground that require relatively low frequency access to a relational database. Serverless Aurora should dramatically reduce the cost of relational database services.

    In the future the reserved RDS/Postgres instance currently used by nderground may be replaced by Serverless Aurora. Topstone Software Consulting will also make use of this Serverless Aurora in future web applications.

  • Simple Storage Service (S3)

    S3 is Amazon's highly reliable web storage service. The photographs and text that nderground stores in S3 can be logically referenced via a directory like storage path (e.g., "myDirectory/photos/myphoto.jpg"). Each nderground user is allocated a randomly named top level "directory" that stores the images and text that they store on nderground. The random top level "directory" names are used to make S3 access more efficient, since S3 references will be randomly distributed.

  • DynamoDB

    DynamoDB is Amazon's "NoSQL" service. DynamoDB is the core database service used by nderground. DynamoDB stores the nderground board posts, threaded comments on board posts, photograph metadata and "housekeeping" information like nderground IP access tracking (which is maintained for security to help identify nderground account compromise).

    Careful design of DynamoDB tables and keys can support extremely fast and inexpensive DynamoDB access. The cost of DynamoDB is based on queries. Unlike the RDS/Postgres service, there is no cost when DynamoDB is not in use.

    DynamoDB is not only an extremely attractive service for nderground but can be leveraged for other Topstone Software Consulting mobile and web applications.

  • Lambda

    Creating thumbnail and scaled versions of the photographs that are downloaded to nderground is a memory and compute intensive operation. Before Lambda was available, nderground was configured with extra EC2 resources to avoid the latency when new EC2 instances were added via Elastic Beanstalk scaling. When Lambda became available, photograph processing was moved to Lambda, which can scale rapidly. This allowed nderground to run with fewer pre-allocated EC2 resources.

    The Lambda service comes with free-tier" processing. This includes 1 million requests and (for nderground's Lambda configuration) 3,200,00 seconds of processing. This is 37 days of continuous processing time. This means that for nderground, Lambda processing is, essentially, free.

    A discussion of Lambda photograph processing can be found in this Topstone Software publication.

  • Simple Email Service (SES)

    nderground generates email for a variety of events, including invitations to nderground and notifications to Karass members about board and gallery posts. SES is a managed email service that allows email to be sent without configuring an email server.

    As with Amazon Lambda, the pricing for SES is extremely attractive for nderground. Currently, the first 62,0000 emails per month sent from an Amazon EC2 application are free. Additional emails are $0.10/1,000. This means that even when nderground is heavily used, the cost for the email service will be far lower than the cost of deploying an email server.

Amazon Cloud Architecture, Spring and Elasticsearch Consulting

Topstone Software has extensive experience building scalable web applications on Amazon Web Services. We can help you design your AWS application architecture to provide scalability and optimized cost.

At Topstone Software we have experience building Spring framework applications that utilize a variety of AWS services. These include Elasticsearch, DynamoDB, S3, the Simple Email System and Elastic Beanstalk. We can provide the consulting help to speed your application development or we can develop applications to your specification.