Leveraging History for Faster Sampling of Online Social Networks

Transcription

1 Leeraging History for Faster Sampling of Online Social Netorks Zhojie Zho George Washington Uniersity Nan Zhang George Washington Uniersity Gatam Das Uniersity of Texas at Arlington STRACT With a ast amont of data aailable on online social netorks, ho to enable efficient analytics oer sch data has been an increasingly important research problem. Gien the sheer size of sch social netorks, many existing stdies resort to sampling technies that dra random nodes from an online social netork throgh its restrictie eb/api interface. While these stdies differ idely in analytics tasks spported and algorithmic design, almost all of them se the exact same nderlying technie of random alk - a Marko Chain Monte Carlo based method hich iteratiely transits from one node to its random neighbor. Random alk fits natrally ith this problem becase, for most online social netorks, the only ery e can isse throgh the interface is to retriee the neighbors of a gien node (i.e., no access to the fll graph topology). A problem ith random alks, hoeer, is the brn-in period hich reires a large nmber of transitions/eries before the sampling distribtion conerges to a stationary ale that enables the draing of samples in a statistically alid manner. In this paper, e consider a noel problem of speeding p the fndamental design of random alks (i.e., redcing the nmber of eries it reires) ithot changing the stationary distribtion it achiees - thereby enabling a more efficient drop-in replacement for existing sampling-based analytics technies oer online social netorks. Technically, or main idea is to leerage the history of random alks to constrct a higher-ordered Marko chain. We deelop to algorithms, Circlated Neighbors and Gropby Neighbors Random Walk ( and ) and rigidly proe that, no matter hat the social netork topology is, and offer better efficiency than baseline random alks hile achieing the same stationary distribtion. We demonstrate throgh extensie experiments on real-orld social netorks and synthetic graphs the speriority of or technies oer the existing ones. 1. INTRODUCTION 1.1 Motiation This ork is licensed nder the Creatie Commons Attribtion- NonCommercial-NoDeris 3.0 Unported License. To ie a copy of this license, isit Obtain permission prior to any se beyond those coered by the license. Contact copyright holder by ing info@ldb.org. Articles from this olme ere inited to present their reslts at the 41st International Conference on Very Large Data ases, Agst 31st - September 4th 2015, Kohala Coast, Haaii. Proceedings of the VLD Endoment, Vol. 8, No. 10 Copyright 2015 VLD Endoment /15/06. With the broad penetration of online social netorks and the mltitde of information they captre, ho to enable a third party 1 to perform efficient analytics of data aailable on social netorks - specifically, to anser global and conditional aggregates sch as SUM, AVG, and COUNT (e.g., the aerage friend cont of all sers liing in Texas) - has become an increasing important research problem in the database commnity [15, 20, 19]. Applications of sch aggregate estimations range from sociology research, nderstanding economic indicators to obsering pblic health trends, bringing benefits to the entire society. Technically, the challenge of data analytics oer online social netorks mainly stems from the limitations of their (eb) ery interfaces aailable to a third party - most online social netorks only allo local neighborhood eries, ith inpt being a node (i.e., ser) and otpt being its immediate neighbors. Sch a ery interface makes retrieing the entire graph topology prohibitiely expensie for a third party (in terms of ery cost - de to the large size of real-orld social netorks). To address the problem, existing stdies resort to a sampling approach, specifically the sampling of nodes (i.e., sers) throgh the restrictie interface of a social netork [22, 24], to enable aggregate estimations based on the sampled nodes. This is also the approach e focs on in the paper. 1.2 Existing Technies and Their Problems While there hae been a ide ariety of social netork sampling designs [12, 10, 11, 7] proposed in the literatre, the ast majority of them share the same core design: random alk, a Marko Chain Monte Carlo (MCMC) method hich iteratiely transits from one node to a random neighbor. The ariations differ in their design of the transition probabilities, i.e., the probability distribtions for choosing the random neighbor, hich are natrally adjsted for the arios types of analytical tasks they target. Nonetheless, the core random alk design remains the same - after performing the random alk for a nmber of steps, the crrent node is taken as a sample - and, after repeated exections of random alks, e can generate (statistically accrate) aggregate estimation from the mltiple sample nodes. To nderstand the problem ith this core design, it is important to obsere the key bottleneck of its practical implementations. Note that sch a sampling design incrs to types of oerhead: One is ery cost: Each transition in the random alk reires one local neighborhood ery (described aboe) to be issed to the online social netork, hile almost all real-orld social netorks enforce rigid ery-rate limits (e.g., Titter allos only 15 local neighborhood eries eery 15 mintes). The second type is the local processing (time and space) oerhead for recording sampled nodes and 1 i.e., one ho is not the social netork oner - examples inclde sociologists, economists, etc. 1034

2 compting aggregate estimations. One can see that the bottleneck here is clearly the ery cost - compared ith the extremely lo ery rate alloed by online social netorks (e.g., 1 minte/ery for Titter), the local processing oerhead (linear to the sample size [13]) is negligible. With the nderstanding that a sampling algorithm mst minimize its ery cost, the problem ith existing technies can be smmarized in one sentence: The core random alk design reires a long brn-in before a node can be taken as a sample - since each step reires one ery, this leads to a high ery cost and therefore a ery slo sampling process oer real-orld online social netorks. To nderstand hat the brn-in period is and hy it is reired, e note that a sample node can only be sed for analytics if e kno the bias of the sampling process, i.e., the probability for a node to be taken as a sample. The reason for this reirement is simple - only ith knoledge of the sampling bias can e properly correct it to ensre eal representation of all applicable tples. Nonetheless, actally learning the sampling distribtion ithot knoledge of the global graph topology is difficlt. A desirable property of random alk is that it asymptotically conerges (as it gros longer) to a stationary sampling distribtion that can be deried ithot knoledge of the graph topology (e.g., ith probability proportional to a node s degree for simple random alk [18], or niform distribtion for Metropolis-Hastings random alk [18]). The nmber of steps reired for a random alk to reach this stationary distribtion is the brn-in period hich, nfortnately, is often ite long for real-orld social netorks [24, 7]. 1.3 Or Idea: History-Aare Random Walks The focs of this paper is to offer a drop-in replacement for this core design (of random alk), sch that existing samplingbased analytics technies oer online social netorks, no matter hich analytics tasks they spport or graph topologies they target, can hae a better efficiency ithot changing other parts of their design. To do so, e shorten the brn-in period (i.e., redce the ery cost) of the fndamental random-alk design ithot changing the stationary distribtion it achiees - ensring the transparency of this change to ho the core design is called pon in data analytics. Technically, or key idea here is motiated by an obseration on the potential aste of eries cased by the existing random alk design: Most existing random alk based technies are Marko Chain Monte Carlo (MCMC) methods that are memoryless - i.e., they do not take into accont the historic nodes encontered in a random alk in the design of ftre transitions. These methods, hile simple, aste sbstantial chances for leeraging historic eries to speed p random alks - a aste e aim to eliminate ith or proposed ideas. Or main idea deeloped in the paper is to introdce historic dependency to the design of random alks oer graphs. Specifically, e consider the nodes already isited by the crrent random alk hile deciding here to go for the next step. We start ith deeloping a simple algorithm called Circlated Neighbors Random Walk (). The difference beteen and the traditional random alk is sampling ith and ithot replacement hen deciding on the next moe. Specifically, consider a random alk ith the last moe being. To determine the next moe, the traditional random alk design is to sample niformly at random from the neighbors of. With, this sampling is done ith replacement - in other ords, if the random alk has transmitted throgh before, then e exclde from being considered for the next moe, ntil the random alk has passed throgh x for all neighbors x of. We proe that, hile shares the exact same stationary distribtion (see Definition 1) as the traditional simple random alk, it is proably more (or eally) efficient than the traditional random alk no matter hat the nderlying topology is. A rationale behind the design of can be explained ith the folloing simple example. Note that if a random alk oer a large graph comes back to after jst passing thogh, it is likely an indication that leads to a small component that is not ell connected to the rest of the graph (as otherise the probability of going back to shold be ery small). Falling into sch a small component trap is ndesirable for a random alk, hich needs to spread to the entire graph as ickly as possible. Ths, an intitie idea to improe the efficiency of a random alk is to aoid folloing again, so as to increase the chance of folloing the other (hopeflly better-for-random-alk) edges associated ith. This intition leads to the circlated, ithot-replacement, design of 2. ased on the idea of, e deelop Gropy Neighbors Random Walk (), hich frther improes the performance of a random alk by considering not only the nodes isited by a random alk, bt also the obsered attribte ales of these isited nodes. To nderstand the key idea of, consider the folloing intitie obseration on the strctre of an online social netork: sers ith similar attribte ales (e.g., age, interests, occpation) are more likely to be connected ith each other. Leeraging this obseration, to decide hich neighbor of to transit to from, partitions all neighbors of into a nmber of strata according to their ales on an attribte of interest (often the measre attribte to be aggregated - see discssions in Section 4.1). Then, first selects a stratm niformly at random ithot replacement - i.e., it circlates among the strata ntil selecting each stratm once. Then, chooses a neighbor of from the selected stratm, again niformly at random ithot replacement. This neighbor then becomes the next step. Similar to the theoretical analysis of, e also proe that shares the same stationary distribtion as the traditional random alk. Intitiely, the design of aims to speed p the random alk by alternating beteen different attribte ales faster, instead of getting stck on a small component of the graph sharing the same (or similar) attribte ale. We shall demonstrate throgh experimental ealation that can significantly improe the efficiency of random alks, especially hen the ltimate objectie of these random alks is to spport aggregate estimations on attribtes sed for stratification in. 1.4 Smmary of Contribtions We smmarize the main contribtions as follos. We consider a noel problem of leeraging historic transitions to speed p random alks oer online social netorks. We deelop Algorithm hich featres a key change from traditional random alks: instead of selecting the next transition by sampling ith replacement from all neighbors of the crrent nodes, performs the sampling ithot replacement, ths becoming a history-dependent random alk. We deelop Algorithm hich frther improes the efficiency of random alk by leeraging not only the history of nodes isited by a random alk, bt also the obsered attribte ales of these isited nodes. 2 One might onder hy the circlation is conditioned pon traeling throgh an edge (i.e., ) again rather than traeling throgh a ertex (say ) again - this is indeed a sbtle point hich e frther address in Section

3 We present theoretical analysis hich shos that, hile and both prodce samples of the exact same distribtion as the traditional random alk, they are proably more efficient no matter hat the nderlying graph topology is. Or contribtion also incldes extensie experiments oer real-orld online social netorks hich demonstrate the speriority of and oer traditional random alk technies. 1.5 Paper Organization This paper is organized as follos. Section 2 describes preliminaries of random alks. Section 3 and section 4 introdce Circlated Neighbors Random Walk () and Gropby Neighbors Random Walk (). Section 5 is abot to discssions of or algorithms. Section 6 shos the reslts of or experiments. Section 7 oeries the related orks. Section 8 proides or brief conclsion. 2. PRELIMINARIES In this section, e introdce preliminaries that are important for the technical deelopment of this paper. Specifically, e start ith introdcing the access model for online social netorks, folloed by a discssion of the most poplar method of sampling an online social netork - random alks. Specifically, e shall introdce to important concepts related to random alks, order and stationary distribtion of a Marko chain, hich are critical for the deelopment of or technies later in the paper. Finally, at the end of this section, e define the key performance measres for a random alk based sampling algorithm. 2.1 Access Model for Online Social Netorks As a third party ith no direct access to the backend data repository of an online social netork, the only access channels e hae oer the data is the eb and/or API interface proided by the online social netork. While the design of sch interfaces aries across different real-orld online social netorks, almost all of them spport eries that take any ser ID as inpt and retrn to types of information abot : N(), the set of all neighbors of, and all other attribtes of (e.g., ser self-description, profile, posts). Note that the definition of a neighbor may differ for different types of social netorks - e.g., Titter distingishes beteen folloers and folloees and ths featres directed connections and asymmetric neighborhoods (i.e., a neighbor of may not hae in its neighbor s list), hile Google Pls featres ndirected friendship edges and ths symmetric neighborhoods. For the prpose of this paper, e consider ndirected edges - i.e., N(), there is N(). Note that online social netorks that featre directed edges can be casted into this ndirected definition. For example, e can define an ndirected edge e if either edge or exists. Gien the definition of node and neighbors, e consider the social netork topology as an ndirected graph G(V, E). V is the set of all the sers (i.e. ertices/nodes) and E is the set of all the connections beteen to sers. E = {e, V }, here e is the connection beteen ser and ser. Also, e se k to denote the degree of node, here k = N(). Many real-orld online social netorks also impose a ery rate limitation. For example, Titter recently pdate their API rate limits to 15 calls eery 15 mintes 3. Yelp offers only 25,000 API 3 calls per day Random alk Almost all-existing orks on sampling an online social netork throgh its restrictie eb/api interface (as modeled aboe) are ariations of the random alk technies. We briefly discss the general concept of a random alk and a specific instance, the simple random alk, respectiely as follos Random Walk as an MCMC Process Order: Intitiely, a random alk on an online social netork randomly transits from one node to another according to a predetermined randomized transition algorithm (and the neighborhood/node information it has retrieed throgh the restrictie interface of the online social netork). From a mathematical standpoint, a random alk can be considered a Marko Chain Monte Carlo (MCMC) process ith its state X i being the node isited by the random alk at Step i and transition probability distribtion Pr(X n = x n X n 1 = x n 1,..., X 1 = x 1) = Pr(X n = x n X n 1 = x n 1,..., X n m = x n m) here x i V. Here m (1 m < n) is the order of the Marko Chain. Most existing random alk technies oer online social netorks are first-ordered (i.e., ith m = 1). That is, ith these random alks, the next transition only depends pon the crrent node being isited, and is independent of the preios history of the random alk. For higher-order random alks, the next transition is determined by not only the crrent node, bt also an additional m 1 steps into the history as ell (e.g., the non-backtracking simple random alk (N-) [11] has an order of m = 2, as it aoids transitioning back to the immediate last node heneer possible). Note that the random alk technie e ill present in the paper has a mch higher order than these existing technies - more details in Section 3 and Section 4. Stationary Distribtion: The premise of sing random alk to generate statistically alid samples is that, after performing a random alk for a sfficient nmber of steps, the probability for the random alk to land on each node conerges to a stationary distribtion π. In other ords, after sfficient nmber of steps - T steps, if the random alk has stationary distribtion, then the probability of staying at node after T steps is the same as after T + 1 steps. The conergence has been proed to hold as long as the transition is irredcible and positie recrrent [16]. Usally a Simple Random Walk (defined in section 2.2.2) on a connected non-bipartite graph has a stationary distribtion. DEFINITION 1. [Stationary distribtion]. Let π j denote the long rn proportion of time that a random alk spends on node j [16]: 1 π j = lim n n n P r{x m = j}, (1) m=1 One can see that the stationary distribtion of a random alk determines ho e can deal ith the samples, and it enables s to frther analyze the samples like estimating certain aggregates Simple Random Walk In the crrent literatre of sampling online social netorks throgh their restrictie interfaces, Simple Random Walk () is one of the most poplar technies being sed. Simple random alk is an

4 order-1 Marko chain ith transition distribtion being niform on all nodes in the neighborhood of the crrent node x i. Formally, e hae the folloing definition. DEFINITION 2. [Simple Random Walk]. Gien graph G(V, E), and a node V, a random alk is called Simple Random Walk if it chooses niformly at random a neighboring node N() and transit to in the next step. { 1/k if N(), P = (2) 0 otherise. Corresponding to this transition design, one can easily compte the stationary distribtion for simple random alk as π = k 2 E That is, selects each node in the graph ith probability proportional to its degree. In later part of the paper, e shall sho ho or proposed technie achiees the same sampling distribtion hile significantly improing the efficiency of random alks. 2.3 Performance Measres for Sampling There are to important objecties for sampling from an online social netork: minimizing the sampling bias and minimizing the ery cost. We define the to performance measres corresponding to the to objecties respectiely as follos. Sampling ias: Intitiely, sampling bias measres the distance beteen the stationary distribtion of a random alk and the actal sampling distribtion achieed by the actal exection of the random alk algorithm. To nderstand hy this is an important measre, e note that there is an inherent tradeoff beteen sch a distance and the ery cost reired for sampling: If one does not care abot the sampling bias and opts to stop a random alk right here it starts, the sampling bias ill be the distance beteen the stationary distribtion and a probability distribtion ector of the form {1, 0,..., 0} (i.e., hile the starting node has probability 100% to be sampled, the other nodes hae 0%) - i.e., an extremely large sampling bias. We propose to measre sampling bias ith three metrics for arios prposes in this paper: KL-diergence [5], l 2-distance [5], and a golden measre of the error of aggregate estimations prodced by the sample nodes. Specifically, hile the former to measres can be sed in theoretical analysis and experimental analysis on small graphs, they are infeasible to compte for large graphs sed in or experimental stdies - motiating s to se the last measre as ell. We shall frther discss the bias and the accracy of arios random alk algorithms in the experiments section 6. Qery Cost. Another key performance measre for sampling oer an online social netork is the ery cost - i.e., the nmber of eries (as defined in the access model described in Section 2.1) one has to isse in order to obtain a sample node. Note that ery cost here is defined as the nmber of nie eries reired, as any dplicate ery can be immediately retrieed from local cache ithot consming the aforementioned ery rate limit enforced by the online social netork. Asymptotic Variance. Variance is one of the most important things to jstify the performance of random alk algorithms. Of corse, e can measre the sampling bias gien certain ery cost, bt after sfficient nmber of steps, the efficiency of an estimator proided by random alk samples is directly tied to the ariance, hich is also important for s to theoretically compare the performance of random alks. For the prpose of this paper, e adopt the commonly sed asymptotic ariance in Marko Chain. (3) DEFINITION 3. [Asymptotic Variance]. Gien a fnction f( ) and an estimator ˆµ n = 1 n n t=1 f(xt), the asymptotic ariance of the estimator is defined as V (ˆµ) = lim nvar(ˆµn) (4) n Note that this does not depend on the initial distribtion for X 0. In addition, in practice, e old se an estimator based on only X t ith t greater than some ery large nmber - say h - that e beliee the chain has reached a distribtion close to its stationary distribtion π after h steps. 3. : CIRCULATED NEIGHORS RANDOM WALK In this section, e deelop Circlated Neighbors Random Walk (), or first main idea for introdcing historic dependency to the design of random alks oer graphs. Specifically, e start ith describing the key idea of, folloed by a theoretical analysis of (1) its eialence ith simple random alk () in terms of (stationary) sampling distribtion, and (2) its speriority oer on sampling efficiency. Finally, e present the psedo code for Algorithm. 3.1 Main Idea and Jstification Key Idea of : In traditional random alks, the transition at each node is memoryless - i.e., hen the random alk arries at a node, no matter here the alk comes from (i.e., hat the incoming edge is) or hich nodes the alk has isited, the otgoing edge is alays chosen niformly at random from all edges attached to the node. The key idea of is to replace sch a memoryless transition to a statefl process. Specifically, gien the preios transition of the random alk, instead of selecting the next node to isit by sampling ith replacement from N(), i.e., the neighbors of, e perform sch sampling ithot replacement. Sch a change is demonstrated throgh an example in Figre 1: When a transits throgh for the first time, it selects the next node to isit in the same ay as traditional random alk - i.e., by choosing niformly at random from N(). Nonetheless, if the random alk transits throgh again in the ftre, instead of selecting the next node from N(), e limit the choice to be from N() {}. One can see that, gien a transition, or selection of the next node to isit is a process of sampling ithot replacement from N(). This, of corse, contines ntil N(), the random alk has passed throgh, at hich time e reset memory and restart the process of sampling ithot replacement. Also, e introdce a notation b(, ), hich is defined as a set of nodes in N() that e hae passed throgh. Ths, e generalize the idea of as: 1. Each time hen the random alk traels from to, e niformly choose the next candidate node from N() b(, ). 2. Let b(, ) b(, ) {}. If b(, ) = N(), let b(, ) =. Intitie Jstification: To nderstand hy improes the efficiency of the sampling process, e start ith an intitie explanation before presenting in the next sbsection a rigid theoretical proof of s speriority oer the traditional simple random alk () algorithm. Intitiely, the jstification for is ite straightforard: If a random alk traels back to after only a small nmber 1037

5 A (a) N() (b) AAN() {} A O (c) N() {, } (d) N() {,, } (e) N() A -> A -> A -> A -> Figre 1: Demo of, it chooses the next candidate from the set N() b(, ). AA A A of steps, it means that the choice of is not a good one for sampling becase, ideally, e old like the random A alk path to propagate to all parts of graph as ickly as possible, instead of being stck at a small, strongly connected AA sbgraph A O like one that incldes, and. As sch, it is a natral choicefor to aoid folloing again in the next transition, bt to instead choose a node from N() {} niformly at random. 3.2 Theoretical Analysis In this section, e ill first introdce the concept of path blocks and then theoretically sho that and hae the same stationary distribtion - ith the fndamental difference beteen the to being the ho path blocks alternate dring the random alk process. Finally, e proe that prodces more accrate samples (i.e. ith smaller or eal asymptotic ariance than ) regardless the graph topology. S3 Path blocks: Intitiely, path blocks diide a random alk s path into consectie segments. Formally, e hae the definition of path blocks: DEFINITION 4. [Path blocks]. Gien a random alk oer graph G(V, E), denote its path as X 0, X 1,..., X m,..., here X i V. A path block ij is defined as ij = {X i, X i+1,..., X j}, j > i. (5) An interesting obseration critical for the design of is that for path block ij, X i can be the same as X j, hich is called recrrence in a Marko Chain [16]. As a typical positie recrrent Marko Chain, ill alays go back to the same node if the nmber of steps is sfficiently large. Formally, positie recrrence means that if e denote M i = n=0 I{Xn = i X0 = i}, i V, then its expected ale E(M i) is alays finite. The reason hy recrrence is an important concept for is becase, as one can see from its on, the key difference beteen and is ho e select the next transition hen a recrrence happens. As mentioned in the introdction, for the prpose of design, one actally has to choices on ho to define a recrrence - based on edge-based and node-based path blocks, respectiely: Edge-based: recrrence if X i = X j 1, X i+1 = X j (6) Node-based: recrrence if X i = X j (7) In (and the folloing discssions in ), e choose the edge-based design for the folloing main reason: Note that ith the edge-based definition, path blocks separated by recrrences hae mch longer expected length than ith the node-based design, becase it takes mch more steps for a random alk to trael back to an edge than a node. As sch, edge-based path blocks tend to hae similar distribtion for each block (proed in Theorem 1), leading to a smaller inter-pathblock ariance, hich in trn reslts in a more significant redction of asymptotic ariance broght by S2 Sampling Distribtion: As discssed in Section 2.2, for, the () S3 S2 DEFINITION 5. [Path blocks (i) rooted on edge e ]. Gien () Figre 2: Path blocks in s ithot-replacement design (proed in Theorem 2). We also condcted extensie experiments to erify the speriority of edge-based design - thogh the reslts are not inclded in this paper de to space limitations. sampling distribtion is π j = k j/2 E., on the other hand, is a higher-order Marko Chain. As sch, the sampling distribtion of is calclated as in eation (1). We ill introdce a specific kind of path blocks first. a random alk ith sbseence i 1 2 h here i, if there does not exist j [1, h 1] ith i = and i+1 =, then e call the prefix of this sbseence ending on h - i.e., i 1 2 h an instance of path block (i) rooted on e. What e really mean here is that once a random alk reaches for the first time, the remaining random alk can be partitioned into consectie, non-oerlapping path blocks rooted on edge e, each of hich starts ith and ends ith the last node isited by the random alk before isiting again. Using this definition, the folloing theorem indicates the eialence of sampling distribtions of and. THEOREM 1. Gien a graph G(V, E), has the stationary distribtion π() = k /2 E. PROOF. The proof has been proided in technical report [23]. One can see from the theorem that, for both and, the probability for a node to be sampled is proportional to its degree. As sch, can be readily sed as a drop-in replacement of. Improement on Sampling Efficiency: Haing proed that and prodce the same (distribtion of) samples - e no deelop theoretical analysis on the efficiency of and its comparison ith. Note that, since the sampling distribtions of both and conerge asymptotically to the stationary distribtion (i.e., probability proportional to a node s degree), for a 1038

6 fair comparison beteen the efficiency of these to algorithms, e mst first establish a measre of the distance beteen the sampling distribtion achieed at h steps and the (eental) stationary distribtion. The reason is that, only hen sch a distance measre is defined, e can then compare the nmber of steps each algorithm reires to redce the distance belo a gien threshold ɛ - apparently, the one hich reires feer steps achiees better sampling efficiency. For the prpose of this paper, e follo the commonly sed asymptotic ariance defined in section 2.3. While there are mltiple ays to nderstand the physical meaning of this measre, a simple one is that it measres the mean sare error (MSE) one old get for an AVG aggregate (on the measre fnction f( ) sed in the definition) by taking into accont the nodes encontered in the first h steps of a random alk. One can see that, the loer this MSE is, the closer the sampling distribtion at Step h is to the stationary distribtion. With the asymptotic ariance measre, e are no ready to compare the sampling efficiency of and. The folloing theorem shos that for any measre fnction f( ) and any ale of h (i.e., the nmber of steps), alays achiees a loer or eal ariance - i.e., better or eal sampling efficiency - than. efore presenting the rigid proof, e first briefly discss the intition behind this proof as follos. The proof is constrcted throgh indction. As sch, e first consider the impact of on changing the transition after one edge, i.e., as in the aboe rnning example, the transition after. A key concept sed in the proof is a segmentation of a long random alk into segments according to. Specifically, eery segment of the random alk except the first one starts and ends ith. Figre 3 shos an example of this segmentation. We can see that are generating segments containing alternating path blocks. With the segmentation, a key idea of or proof is to introdce an encoding of each segment according to the first node it isits after. For example, in the case shon in Figre 3 here has three neighbors {,, }, e hae three possible path blocks: {(), (), ()}. It is important to note that, since e are only considering the change after at this time, eery path block can be considered as being dran from the exact same distribtion no matter if or is sed - becase does not make any changes once the first node after, i.e., the code, is determined, ntil the next time is isited. This obseration enables s to simple consider and as seences of codes (i.e. e can map path blocks into codes) in the efficiency comparison - as hat happens ithin each path block is anyay obliios to hether or is being sed. No stdying the seence of codes for and, e make a straightforard yet interesting obseration: gien a seence containing {(), (), ()} ith length h, the nmber of occrrences of (), (), and () ill be the same (at least ithin ±1 range) in, becase they ill alternately appear eery three codes. On the other hand, ith, the nmber of occrrences for {(), (), ()} is statistically eal bt hae an inherent ariance in practice de to randomness. The elimination of this randomness/ariance is exactly hy tends to generate samples ith a smaller ariance than - as rigidly shon in the folloing theorem. THEOREM 2. Gien a graph G(V, E), any property fnction f, and the folloing to estimators for µ based on (ˆµ) and ( ˆµ ): ˆµ n = 1 n n f(x t), t=1 ˆµ n = 1 n n f(x t), (8) t=1 then ill hae no greater asymptotic ariance (defined in Section 2.3) than V ( ˆµ ) V (ˆµ). (9) PROOF. The proof has been proided in technical report [23]. Theorem 2 establishes s speriority hile the folloing Theorem 3 shos ho significant the speriority oer a concrete example: barbell graph. The probability of propagating from one sbgraph to another in is mch greater than the one in. THEOREM 3. Gien a barbell graph G, hich contains to complete sbgraphs G 1, G 2. If e choose the initial node G 1, then the expected ale of t and t satisfy that P > G1 ln G1 (10) P G 1 1 here P and P are the probability for and to trael from G 1 to G 2. PROOF. The proof has been proided in technical report [23]. 3.3 Algorithm Algorithm 1 Circlated Neighbors Random Walk /* Gien x 0 =, x 1 =, b(x 0, x 1) = */ /* Fnction b(, ) can be implemented as a HashMap, and their initial ale are all. */ for i = 2 sample size do /* S denotes the next possible candidates */ S N(x i 1) b(x i 2, x i 1) if S then x i niformly choose a node from S b(x i 2, x i 1) = b(x i 2, x i 1) {x i} else x i niformly choose a node from N(x i 1) b(x i 2, x i 1) = end if end for Algorithm implementation. Algorithm 1 depicts the psedo code for Algorithm. We note that the only data strctre e maintain (beyond hat is reired for ) is a historic hash map of otgoing transitions b(, ) for each edge e e pass throgh - i.e., if the random alk has passed throgh edge e before, then b(, ) contains the neighbors of hich hae been chosen preiosly (dring the alk) as the otgoing transitions from. Time and space complexity. reires a hash map that continosly records the otgoing transitions for each edge, i.e. the key-ale pair: e b(, ). Assme alks K steps, and the keys e are niformly distribted among all the edges in the graph for (the same as ), then this hash map as a hole occpies expected O(K) space (according to [6] if e are sing dynamic perfect hashing). Also, each step s amortized expect time complexity is O(1), so the total expected time complexity for K steps is O(K). 1039

7 A -> A -> A -> A -> AA A O () () () () () () () () () () Segments of alternating path blocks. They are circlated ith a period of 3. () () () () () () () () () () A A O () AA Path blocks can be consectie in. A A -> A -> A -> A -> Figre 3: Comparison of the block distribtion in and. AA A A O A O RANDOM WALK S2 4.1 asic Idea Recall from Section 3.1 the main idea of : Gien an incoming transition, e essentially circlate the next transition among the neighbors of, ensring that e do not attempt the () () () () () () () () () () Segments of alternating S3 S2 path blocks. They are circlated ith a period of 3. same neighbor tice before enmerating eery neighbor. The key () () () () () () () () () Path blocks can be consectie in. S2 S3 Figre 4: An example of partitioning a node s neighbors into 3 grops. 4. : GROUPY NEIGHORS idea of is a natral extension: Instead of performing the circlation at the granlarity of each neighbor (of ), e propose to first stratify the neighbors of into grops, and then circlate the selection among all grops. In Figre 4, for example, if e isit for the second time, ith the last chosen transition being from to a(nother) node in S 2, then this time e ill randomly pick one grop from S 1 and S 3 (ith probability proportional to the nmber of not-yet-attempted transitions in each grop), and then pick a node from the chosen grop niformly at random. can be smmarized as: 1. It has a global gropby fnction g( ) that ill partition a node s neighbors into disjoint grops. For example, g(n()) = {S 1, S 2,..., S m}. 2. Each time hen the random alk traels from to, e choose the next candidate grop S i from N() S(, ) ith probability S i / N() S(, ), here S(, ) is a set of grops e hae accessed before. 3. Within S i, e niformly at random choose the next candidate node from S i b Si (, ), here b Si (, ) is defined similar to s - ith a range limited in S i. 4. Let b(, ) b(, ) {}, and S(, ) S(, ) {S i}. If b(, ) = N(), let b(, ) =. If S(, ) contains all the possible grops, let S(, ) =. One can see from the design of that, like, it does not alter the stationary distribtion of simple random alk S3 () () - no matter ho the groping strategy is designed. Also, jst like, garantees a smaller or eal asymptotic ariance than. This is formalized in the folloing theorem, the proof of hich can be constrcted in analogy to Theorem 4. THEOREM 4. Gien a graph G(V, E), has the stationary distribtion π() = k S3 S2 /2 E. And for any property fnction f, the folloing to estimators for µ based on (ˆµ) and ( ˆµ ): ˆµ n = 1 n f(x t), ˆµ n = 1 n f(xt ), (11) n n t=1 A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A -> A A -> -> -> -> -> -> A A -> A A -> A A -> A A -> -> A A -> A A -> A A -> A -> A -> A -> A -> A -> In the folloing discssions, e first A -> describe A -> the A rationale -> A be-, and then discss the design of the groping strategy -> A -> A -> hind (for the neighbors of ), in other ords, the design of gropby fnction g(n()) = {S 1, S 2,..., S m}. To nderstand the rationale, A -> -> -> -> -> -> A e start by considering () to extremes of the groping-strategy design. At one extreme is to grop neighbors of in a completely random fashion. With this strategy, is exactly redced to - i.e., eery neighbor of is circlated, ith the order being a random permtation. At the other extreme is the ideal scenario for - as illstrated in Figre 4 - hen nodes leading to similar path blocks are groped together. The intitie reason hy otperforms in this case is easy to nderstand from Figre 4: Circlating among the three grops, instead of attempting a grop more than once, can make the random alk propagate faster to the entire graph rather than being stck in jst one clster. More formally, the adantage offered by can be explained according to the path-block encoding scheme introdced in Section 3.2. Consider an example in Figre 5 here has 4 neighbors. We encode the path block from each neighbor as random ariables ( 1),..., ( 4). Sppose that neighbors are partitioned into to grops: S 1 : { 1, 2} and S 2 : { 3, 4}. One can see that is like stratified sampling, hile is a process of simple random sampling (ithot replacement). As long as the intra-grop ariance for the to grops is smaller than the poplation ariance, offers oerall a loer asymptotic ariance. To nderstand hy, consider an extreme-case scenario ith zero intra-grop ariance (i.e., ( 1) = ( 2), ( 3) = ( 4)). One can see that, hile achiees zero oerall ariance, still has positie ariance de to the inter-grop ariance (i.e., ( 1) ( 3)). Haing discssed the rationale behind, e no trn or attention to the design of the groping strategy. One can see from t=1 then ill hae no greater asymptotic ariance than V ( ˆµ ) V (ˆµ). (12) PROOF. The proof has been proided in technical report [23]. 1040

8 A -> A -> A -> A -> AA A O 4 S2 S2 S2 S2 S2 S2 S2 3 (1) (3) (2) (4) (2) (3) (1) (4) (2) (4) (1) (3) 1 Segments of alternating grops and path blocks. They are circlated ith a period of 4. 2 Figre 5: A demo of S2 the aboe discssion that the main objectie here is the grop together neighbors that lead to similar path blocks - i.e., random alks starting from nodes in the same grop shold share similar characteristics. We old like to make to obserations for the design: First, locality is a property idely recognized for social netorks - i.e., sers ith similar attribte ales (e.g., age, occpation) tend to hae similar friends (and therefore lead to similar path blocks). Ths, groping neighbors of based on any attribte is likely to otperform the baseline of random grop assignments. Second, hich attribte to se for grop assignments also has an implication on the potential sage of samples for analytical prposes. For example, if one knos beforehand that samples taken from a social netork ill be sed to estimate the aerage age of all sers, then designing the groping strategy based on ser age is an ideal design that ill likely lead to a more accrate estimation of the aerage age. To nderstand hy, note that ith this groping strategy, the random alk is likely to ickly propagate to sers of different age grops, instead of being stck at a tight commnity formed by sers of similar ages. Ths, if one knos an important aggregate ery hich the collected samples ill be sed to estimate, then the groping strategy shold be designed according to the attribte being aggregated in the ery. We shall erify this intition ith experimental reslts in Section 6.2. S3 4.2 Algorithm () Algorithm implementation. Algorithm 2 depicts the psedo code for Algorithm. We note that the data strctres e maintain are to hash maps: S(, ) and b Si (, ). S(, ) is a mapping from (, ) to the crrent set of grops that hae accessed before based on the rote ; b Si (, ) is a mapping from () () (,, S () i) to the crrent set of nodes that hae accessed before based on the rote and the otgoing grop S i. Time and space complexity. reires to hash maps that lated ith a period continosly of 3. records the otgoing grops and edges for each edge, i.e. the key-ale pairs: e S(, ) and (e, S i) b Si (, ). ) () () Similar to, e assme alks K steps, and it also has the amortized expected O(K) time complexity and O(K) space RW. complexity, becase the keys e and (e, S i) are niformly distribted among their possible ales [6]. 5. DISCUSSIONS applied to Non-acktracking Random Walk (N-) It is important to note that the idea of - i.e., changing transition pon isiting an edge from sampling ith replacement to sampling ithot replacement - is an idea that can be applied to any base random alk algorithm, inclding both and N- [11]. For example, if e apply the idea to N-, then the reslting algorithm (say N-) ill ork as follos: Upon isiting, instead of sampling the next node ith replacement from N()\ (like in N-), e old sample it ithot replacement from N()\. Note the difference beteen N- Algorithm 2 Gropby Neighbors Random Walk /* Gien x 0 =, x 1 =, a gropby fnction g( ) */ /* And e assme that all S(, ) and b Si (, ) shold be initialized as. */ for i = 2 sample size do g(n(x i 1) = {S 1, S 2,..., S m} CS {S 1, S 2,..., S m} S(, ) if CS then S i choose a grop ith probability S i / CS U S i b Si (x i 2, x i 1) if U then x i niformly choose a node from U b Si (x i 2, x i 1) = b Si (x i 2, x i 1) {x i} else x i niformly choose a node from S i A -> A -> A -> A -> A -> b Si (x i 2, x i 1) = end if else S i niformly choose a grop from {S 1, S 2,..., S m} U S i b Si (x i 2, x i 1) A -> A -> A -> A -> A -> if U then x i niformly choose a node from U b Si (x i 2, x i 1) = b Si (x i 2, x i 1) A -> {x i} -> -> -> -> else x () i niformly choose a node from S i b Si (x i 2, x i 1) = end if end if end for and the algorithm presented in the paper (hich is based on ): With, the sampling is done oer N() hile ith N-, the sampling is done oer N()\ - indeed a carryoer change from N-. We hae proided detailed proofs in technical report [23] that N- has the same stationary distribtion as N-, and the asymptotic ariance of N- is also less than or eal to s. Ho does graph size affects and. First, e note that the graph size is nlikely to be a main factor in the historic isit probability. To nderstand hy, consider oer an ndirected graph and the probability of going back to the starting node (hich, ithot loss of generality, is the probability for historic isit to occr). Note that the probability of going back to the starting node at Step i keeps decreasing ith i, ntil the random alk isits the starting node again [4]. Ths, the historic isit probability mainly depends on the first k steps after isiting a node, here k is a small constant. In more intitie terms, a random alk is mostly likely to go back to a node only a fe steps after isiting the node (becase, after only a fe steps, the random alk is 1041

9 likely still ithin a tightly connected local neighborhood of the node). This essentially means that the historic isit probability is nlikely to be sensitie to graph size - after all, een hen the graph size tends to infinity, the probability of isiting (or re-isiting) a node ithin a constant nmber of steps is nlikely to change mch. As an extreme-case example, groing the graph beyond the k-hop neighborhood of the starting node does not change the historic isit probability ithin k steps at all. 6. EXPERIMENTS 6.1 Experimental Setp Hardare and platform: We condcted all experiments on a compter ith Intel Core i3 2.27GHz CPU and 64bit Ubnt Linx OS. Datasets: We tested three types of datasets in the experiments: ell-knon pblic benchmark datasets that are small sbsets of real-orld social netorks, large online social netorks sch as Google Pls and Yelp, and synthetic graphs (for demonstrating extreme-case scenarios) - e.g., barbell graphs and small clstered graphs. We briefly describe the three types of datasets e sed respectiely as follos (see the smmary of these datasets in Table 1). Pblic enchmark: The Facebook dataset is a pblic benchmark dataset collected from [1]. It is a preiosly-captred topological snapshot of Facebook and it has been extensiely sed in the literatre (e.g., [14]). Specifically, the graph e sed is from the 1684.edges file. Yotbe is another large pblic benchmark graph collected from [21]. In these pblic benchmark dataset, e simlated a restricted-access eb interface precisely according to the definition in Section 2.1, and ran or algorithms oer the simlated interface. Large Online Social Graphs: Google Pls 5. To test the scalability of or algorithms oer a large graph, e performed experiments oer a large graph e craled from Google Pls that consists of 240,276 sers. We obsere that the interface proided by Google Social Graph API strictly adheres to or access model discssed in Section i.e., each API reest retrns the local neighborhood of one ser. Yelp dataset 6. We extracted the largest connected sbgraph containing 119,839 sers (ot of 252,898 sers) from the dataset. We restored all the dmped JSON data into MongoD to simlate API reests. Since e focs on sampling ndirected graphs in this paper, for datasets that featre directed graphs, e first conerted it to an ndirected one by only keeping edges that appear in both directions in the original graph. Note by folloing this conersion strategy, e garantee that a random alk oer the ndirected graph can also be performed oer the original directed graph, ith an additional step of erifying the existence of the inerse direction (resp. ) before committing to an edge (resp. ) in the random alk. Synthetic Graphs: We also tested or algorithms oer synthetic graphs, sch as barbell graphs and graphs ith high clstering coefficients for to main prposes: One is to demonstrate the performance of or algorithms oer ill-formed graphs as these synthetic graphs hae ery small condctance (i.e., highly costly for brning in). The other is to control graph parameters sch as nmber of nodes that e cannot directly control oer the aboe-described real-orld graphs. It is important to note that or sage of a theoretical graph generation challenge model does not indicate a belief of the model being a representation of real-orld social netork topology. Algorithms: We implemented and tested fie algorithms in the experiments: Simple Random Walk () [12], Metropolis-Hastings Random Walk (MHRW) [8], Non-acktracking Simple Random Walk (N-) [11] - a state-of-the-art random alk algorithm hich ses an order-2 Marko Chain, and to algorithms proposed in this paper: Circlated Neighbors Random Walk () in Section 3 and Gropby Neighbors Random Walk () in Section 4. For each algorithm, e ran it ith a ery bdget ranging from 20 to 1000, and take the retrned sample nodes to measre their ality (see performance measres described belo). For, e tested arios groping strategies, as elaborated in Section 6.2. It is important to nderstand hy e inclded MHRW in the algorithms for testing. Note that, hile, N- and or to algorithms all share the same (target) sampling distribtion - i.e., each node is sampled ith probability proportional to its degree - MHRW has a different sampling distribtion - i.e., the niform distribtion. Ths, it is impossible to compare the samples retrned by MHRW ith the other algorithms properly. We note that or prpose of inclding MHRW here is to simply erify hat has been recently shon in the literatre [7] and [11] - i.e., for practical prposes sch as aggregate estimation oer social netorks, the performance of MHRW is mch orse than the other based algorithms, jstifying the sage of as a baseline in or design. Performance Measres. Recall from Section 2.3 that a sampling algorithm for online social netorks shold be measred by ery cost and bias - i.e., the distance beteen the actal sampling distribtion and the (ideal) target one, hich in or case is π() = k /(2 E ). To measre the ery cost, one simply conts the nmber of nie eries issed by the sampler. The measrement of bias, on the other hand, reires s to consider to different methods (and three measres) described as follos. For a small graph, e measred bias by rnning the sampler for an extremely long amont of time (long enogh so that each node is sampled mltiple times). We then estimated the sampling distribtion by conting the nmber of times each node is retrieed, and compared this distribtion ith the target distribtion to derie the bias. Their distances are measred in to forms: (1) KLdiergence [5], and (2) l 2-distance [5] beteen the to distribtion ectors. Let P and P sam be the ideal and measred sampling distribtion ectors, respectiely. To measre the distance in KL-diergence, e compte D KL(P P sam) + D KL(P sam P ) here D KL(P Q) = ( ) P (i) ln P ( i), (13) Q( i) i V For l 2-norm, e se P P sam 2. Note that compared ith the KL-diergence based measre, the l 2-norm one is more sensitie to otliers - i.e., large differences on the sampling probability for a single node - hence or sage of both measres in the experiments. For a large graph like Google Pls and Yelp e sed, it is no longer feasible to directly measre the sampling probability distribtion (becase sampling each node mltiple times to obtain a reliable estimation becomes prohibitiely expensie). Ths, e considered another measre of bias, aggregate estimation error, for experiments oer large graphs. Specifically, e first sed the collected samples to estimate an aggregate oer all nodes in the graph - e.g., the aerage degree or reies cont - and then compare the 1042