IN the last decade, vehicular ad-hoc networks (VANETs) Prefetching-Based Data Dissemination in Vehicular Cloud Systems

1 Prefetchin-Based Data Dissemination in Vehicular Cloud Systems Ryansoo Kim, Hyuk Lim, and Bhaskar Krishnamachari Abstract In the last decade, vehicular ad-hoc networks VANETs have been widely studied as an effective method for providin wireless communication connectivity in vehicular transportation systems. In particular, vehicular cloud systems have received abundant interest for the ability to offer a variety of vehicle information services. We consider the data dissemination problem of providin reliable data delivery services from a cloud data center to vehicles throuh roadside wireless access points APs with local data storae. Due to intermittent wireless connectivity and the limited data storae size of roadside wireless APs, the question of how to use the limited resources of the wireless APs is one of the most pressin issues affectin data dissemination efficiency in vehicular cloud systems. In this paper, we devise a vehicle route-based data prefetchin scheme, which maximizes data dissemination success probability in an averae sense when the size of local data storae is limited and wireless connectivity is stochastically unknown. We propose a reedy alorithm and an online learnin alorithm for deterministic and stochastic cases, respectively, to decide how to prefetch a set of data of interest from a data center to roadside wireless APs. Experiment results indicate that the proposed alorithms can achieve efficient data dissemination in a variety of vehicular scenarios. Index Terms Vehicular cloud system, data dissemination, vehicular ad hoc networks, online learnin, reedy alorithm, roadside wireless access point. I. INTRODUCTION IN the last decade, vehicular ad-hoc networks VANETs have been widely studied as a method for incorporatin wireless communication capabilities in vehicular transportation systems for safety, enery, and comfort issues [1]. VANETs consist of two types of nodes, i.e., mobile vehicles and stationary roadside wireless access points APs; the wireless APs serve as an infrastructure for network connectivity in VANETs. In VANETs, vehicle-to-vehicle V2V, infrastructure-to-vehicle I2V, and vehicle-to-infrastructure V2I communications are defined dependin on the direction of traffic flow [2]. While V2V is primarily used for exchanin immediate drivin information amon neihborin vehicles on Copyriht c 213 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sendin a request to pubs-permissions@ieee.or. This research was supported in part by the U.S. National Science Foundation via award number CNS-121726, and by the National Research Foundation of Korea rant funded by the Korea overnment MSIP 214R1A2A2A162. Correspondin author: H. Lim. R. Kim and H. Lim are with the School of Information and Communications, Gwanju Institute of Science and Technoloy GIST, Gwanju 5-712, Republic of Korea. Email: {rskim,hlim}@ist.ac.kr B. Krishnamachari is with the Department of Electrical Enineerin, Viterbi School of Enineerin, University of Southern California, Los Aneles, CA 989, USA. Email: bkrishna@usc.edu the road for safety purposes, I2V and V2I aim at data delivery services from/to the Internet throuh the roadside wireless APs in VANETs. The vehicular cloud system VCS is a new emerin technoloy that can provide cloud services for various vehicle information applications such as multimedia streamin, autonomous naviation services, and remote vehicle dianosis. The infrastructure for VCS consists of hih-performance cloud servers at a data center and a number of roadside wireless APs with limited-sized local data storae. If an application requires hih computational power or an extensive amount of data storae, it is more desirable to be implemented and executed as a cloud service of VCS because vehicles may have insufficient data processin and storae capability to run such a heavy application in a stand-alone manner [3]. When a vehicle needs data delivery and computin services for vehicle information applications, it uses the roadside wireless APs to contact the cloud servers. Moreover, the cloud server can perform computationally intensive tasks and disseminate output data to the vehicular subscriber. While the roadside APs are connected to the cloud servers throuh wired links, the connection between the vehicles and the wireless APs is intermittently available as the vehicle enters and leaves the service coverae areas of the wireless APs. Thus, owin to vehicle mobility and intermittent connectivity, data dissemination to mobile vehicles throuh the roadside wireless APs is a challenin problem for successful implementation of vehicular cloud systems [4]. We concentrate our attention on the problem of how to exploit the local data storaes of roadside wireless APs for efficient data dissemination within the VCS. For illustration, we consider a data dissemination scenario for VCS as shown in Fiure 1. Suppose a vehicle requests cloud data and the routin path is determined in advance. All of the wireless APs located on the path of the vehicle fetch the data from the cloud servers and attempt to transmit it to the vehicle when the wireless link to the vehicle is established. This approach can achieve the hihest data delivery success probability, but it is not practically applicable because the size of local storae in each wireless AP is too small to store all of the data requested by multiple vehicles at the same time. Note that the number of data chunks that can be prefetched from a cloud server at a sinle time point is limited by the wired link capacity of wireless AP, despite the fact that the AP has a lare storae capacity. For example, consider a multimedia streamin service. If a wireless AP has a iabit Ethernet link with 128 GB solid-state drive SSD storae, it can prefetch 75 1-MB video clips every minute and can store 2,56 video clips. In such a case, with respect to dissemination

2 efficiency, it is more desirable to fetch the data requested by more than two vehicles to the wireless APs located on the intersection of multiple paths rather than that requested by only one vehicle. For example, if the vehicles A and B request the same vehicular cloud service as shown in Fiure 1, it is desirable that AP 5 has the data chunk and transmits it to the vehicles because both vehicles pass by AP 5. It is also important to consider that vehicles have connectivity to wireless APs only when they stay for at least a certain time interval within the service coverae area of the roadside APs; moreover, the connectivity is susceptible to the time-varyin wireless channel. In this paper, we devise a vehicle route-based data prefetch framework for data dissemination in vehicular cloud system, which maximizes the areate dissemination success probability in an averae sense when the size of local data storae is limited and wireless connectivity is stochastically unknown. We formulate this data dissemination as a binary optimization problem, for which the optimal solution can be obtained by a deterministic combinatorial alorithm. We propose a reedy alorithm and analyze its approximate worstcase performance bound. We also propose an online learnin alorithm based on multi-armed bandits MABs to maximize the areate dissemination success probability in an averae sense by capturin the unknown stochastic characteristics of wireless connectivity. The proposed data dissemination scheme is applicable to delay-tolerant vehicular data services such as entertainment content distribution, naviation data updates, and online travel uide services. The rest of this paper is oranized as follows. First, we present a survey of related work in Section II. In Section III, we describe the system models and assumptions for data dissemination of vehicular cloud systems. In Section IV, we formulate a data dissemination problem that maximizes the areate dissemination success probability and propose a reedy alorithm and an online learnin alorithm for deterministic and stochastic cases, respectively. In Section V, we show numerical experiment results followed by conclusions in Section VI. Fi. 1. Vehicle route and roadside wireless AP based data dissemination scenario for a vehicular cloud system. II. RELATED WORK The data dissemination research for VANETs is summarized into the two cateories of V2V and V2I/I2V communications. The data dissemination research for V2V communications focuses on how to achieve reliable and timely data delivery amon mobile vehicles on roads over intermittently connected wireless links [5] [1]. In [5], the data pourin DP alorithm with intersection bufferin was proposed. The vehicles at intersections keep the data sent by the source node in their buffers and repeatedly rebroadcast it to other vehicles passin the intersection. In [6], the route information of vehicles, which is readily available throuh the GPS enabled naviation system in the vehicles, is used for alleviatin channel conestion in data dissemination by selectin appropriate routin paths. In [7], the relative distance between neihborin mobile vehicles is predicted and exploited for improvin reliability of data delivery. In [8], Schwartz et al. proposed adaptive network load control for fair data dissemination in VANETs. In [9], Ye et al. studied a peer-to-peer data dissemination problem and proposed a network codin based data broadcastin scheme for improvin data reception efficiency. The dissemination complete time and steady-state data dissemination velocity for the peer-to-peer data dissemination were also mathematically analyzed in [9]. In [1], Sathiamoorthy et al. investiated vehicle-to-vehicle data sharin usin erasure codes for reducin data dissemination latency in vehicular networks. In particular, they focused on the problem of how to store erasure coded data in vehicles to maximize vehicle-to-vehicle collaboration opportunities. The data dissemination for V2I and I2V communications focuses on how to efficiently share the limited resource of roadside APs to improve the quality of data dissemination services. In [11], Lian et al. proposed a cooperative data dissemination approach. At the network level, network resources were cooperatively manaed so as to satisfy the quality of service QoS requirements for realtime and non-realtime traffic. At the packet level, cooperative transmission for the sake of increasin the hih packet transmission rate was proposed. In [12], rateless codin technoloy was applied at roadside wireless APs to improve the efficiency of data dissemination. In I2V data dissemination, two important factors that sinificantly affect the data dissemination performance are the limited buffer size of roadside wireless APs and the intermittent connectivity between the wireless APs and mobile vehicles [13] [16]. In [13], a hybrid data dissemination assisted by static nodes was proposed. When there are no vehicles that can deliver the data alon a routin path, static nodes located at road intersections keep the data and forward it when the routin path becomes available. In [14], wireless transmission characteristics for sendin and receivin lare amounts of data from a movin vehicle to the roadside wireless APs were investiated empirically. In [15] and [16], a wireless measurement study for vehicles under different drivin conditions was

3 carried out. In [17], Jeon et al. proposed an infrastructurebased data dissemination that utilizes the trajectory of the vehicles that the packets with a delay constraint are destined for. As a vehicle is movin alon a pre-determined route path, one of the relay nodes on the path is dynamically selected as the destination for each requested data packet such that the packet delivery delay is minimized while satisfyin the packet reception probability requirement. While the work in [17] mostly focused on data delivery latency rather than data dissemination efficiency for the roadside wireless APs with limited storae capacity, we consider an I2V VANET scenario where there exists a constraint on the storae size of roadside wireless APs and the network connectivity is intermittently available, and propose a deterministic reedy alorithm and an online learnin-based alorithm to achieve hih data dissemination efficiency in the VANETs. The data dissemination method for VCS has some similarities with web cachin strateies adopted for enhancin local access to popular Internet contents via proxy servers. Dependin on user web access patterns and network topoloy, the web cachin strateies aim to find the best place for web proxies on the network, and allow the proxies to cache popular contents for reducin user access latency and the amount of Internet access traffic [18]. In [19], Li et al. studied an optimal placement of web proxies amon candidate sites in a tree-based network topoloy in order to minimize the latency for taret web services. They modeled the optimal placement problem as a dynamic prorammin problem, and the optimal solution could be obtained in polynomial time. In [2], Qui et al. studied an online placement problem of web server replicas under imperfect information about client workload characteristics. They formulated the placement problem as a minimum K-median raph theoretic problem and devised alorithms for minimizin the client cost of accessin data replicated on web servers. In [21], Nimkar et al. focused on how to place multiple copies of video files on distributed proxy servers for video on demand VoD services, and devised reedy video placement and disk load-balancin alorithms. The data dissemination for VCS is also related to the problem of allocatin virtual computin and storae resources in lare-scale distributed systems such as rid and cloud computin environments. In [22], Giuriu et al. dealt with the problem of how to efficiently manae virtual network infrastructures in lare scale data centers while uaranteein resource and availability requirements. To make the optimization problem more tractable, they reduced the searchin space for optimization by specifyin feasible subsets of computin nodes as the candidate sites, and used a raph-based search alorithm for findin the optimal placement of virtual machines. In [23], a multi-objective ant colony system alorithm was adopted to find the optimal placement of virtual machines that can minimize the areate power and resource consumption in cloud infrastructures. Our data dissemination method is similar to web cachin/proxy methods in that both deal with a distributed cache/storae-based data dissemination problem. The distributed multiple cache/storae devices store a certain amount of contents to expedite the service response and to reduce the traffic amount to be downloaded from the central data server. However, there are sinificantly challenin issues that need to be resolved for data dissemination in vehicular network environments: Most web cachin/proxy methods focus on maximizin the hit rate of individual proxy servers for certain content request statistics. Our VCS is formulated such that it uarantees that vehicular subscribers receive the data service successfully from at least one roadside AP while they pass by multiple APs durin their travelin time. Web cachin/proxy methods are usually desined under the assumption that proxy servers provide services to their users throuh a reliable link without sinificant loss of data. In our VCS, the vehicular subscribers communicate with roadside wireless APs throuh a unreliable wireless link. It is assumed that the connectivity is intermittently available, and its distribution is stochastically unknown in advance. The stochastic characteristics of wireless connectivity should be taken into consideration for reliable dissemination services. In these vehicular environments, the drivin routes of vehicles are assumed to be predicted or obtained usin online naviation. This routin information can be exploited for improvin data dissemination performance. For available routin information of vehicular subscribers, our data dissemination method cooperatively manaes the multiple roadside APs on the routes of the subscribers, dependin on their intermittent connectivity distribution and storae capacity, in order to maximize the data service success rate for all data-vehicle pairs. In fact, various existin web cachin/proxy methods could be applied for enhancin the dissemination performance in vehicular network environments because VCS can be considered as a mobile web cachin method that coordinates distributed roadside storaes with network connectivity in vehicular network environment. III. SYSTEM MODEL We consider a VCS that consists of cloud servers at a data center and roadside wireless APs with local data storae. Mobile vehicles have intermittent network connectivity to the cloud system throuh the wireless APs, which are connected to the cloud servers by means of wired infrastructure networks. To expedite data dissemination to vehicles, each AP can prefetch some data from the data center before they are requested from vehicular subscribers. We make the followin assumptions for the VCS: The data in a cloud system is divided into a number of small chunks that are the basic units for data delivery from the data center to vehicles. Each AP is placed at an intersection and has limited transmission coverae such that a vehicle can download data chunks of interest only when it stays within the coverae area for at least a certain amount of time. Each AP has a stochastic characteristic for communicatin with the mobile vehicle oin throuh its coverae

4 TABLE I PARAMETERS USED FOR MODELING VEHICULAR CLOUD SYSTEMS. Parameter w v u b R S θ X P Description Number of wireless APs in the vehicular cloud system Number of vehicles in the area of interest Number of data chunks served by the vehicular cloud system Maximum number of data chunks that can be stored at each local data storae Chunk request matrix R = v u where =1 if the i-th vehicle requests the j-th chunk, and otherwise. Vehicle route matrix S = s i,j v w where s i,j =1 if the i-th vehicle oes throuh the j-th wireless AP, and otherwise. Network connectivity success rate vector θ = θ k w 1 where θ k is an averae rate of successful communications between the k-th AP and mobile vehicles. Binary decision matrix X = x i,j w u where x i,j = 1 if the j-th chunk is to be prefetched to the i-th wireless AP and otherwise. Dissemination failure probability matrix P = p i,j v u where p i,j is a probability that the i-th vehicle fails to download the j-th chunk. reion due to limited communication capacity and timevaryin wireless channels. For effectively usin a data dissemination service, the drivin route of vehicles must be available in advance from online naviation and lon-term archived traces. Each vehicle that uses VCS has to report its drivin route so that the data dissemination alorithm can find the best APs belonin to that route. Note that the selected APs will prefetch data chunks from a cloud server and hold the requested chunks requested by a vehicular subscriber. Whenever the route chanes, it has to be reported to the VCS. If a vehicle does not have a naviation device, its drivin route can be predicted usin historical traces. Given the assumptions outlined above, we first define two matrices of the chunk request matrix R, the vehicle route matrix S, and the network connectivity success rate vectorθ to describe a VCS. Let u, v, and w denote the numbers of chunks, vehicles, and wireless APs, respectively. R = v u is a binary matrix where = 1 when the i-th vehicle requests the j-th chunk, and otherwise. S = s i,k v w is also a binary matrix where s i,k = 1 when the i-th vehicle is expected to o throuh the k-th wireless AP, and otherwise. Last, θ = θ k w 1 is a vector where θ k represents the averae rate of successful communication between the k-th AP and a mobile vehicle passin by the AP. We summarize the parameters used for modelin the VCS in Table I. The data dissemination problem in this paper is to make a decision on which chunks are to be prefetched to the local data storaes of wireless APs to expedite data dissemination in the VCS. Let X = x i,j w u denote a binary decision matrix where x i,j = 1 if the j-th chunk is to be prefetched to the i-th wireless AP, and otherwise. For example, if x i,j = 1 for all i s and j s, it means that all the data chunks are distributed to every local data storae. In this case, vehicles are supposed to successfully receive as many requested chunks as possible, but this may not be efficient because all the chunks are unnecessarily copied to every local storae. In particular, the data storae capacity of roadside wireless APs is limited, and thus all of the data chunks cannot be stored in all the APs. By usin R, S, θ, and X, we derive the dissemination failure probability matrix denoted by P = p i,j v u where p i,j is the probability that the i-th vehicle fails to download the j-th chunk. Then, p i,j is iven by the product of the probabilities that the i-th vehicle fails to download the j-th chunk at every AP that it oes throuh as follows: p i,j = h i,j,k s i,k,θ k,x k,j, 1 where h i,j,k is the probability that the i-th vehicle fails to download the j-th chunk at the k-th AP, and h i,j,k s i,k,θ k,x k,j = 1 s i,k θ k x k,j. IV. PROPOSED DATA PREFETCHING SCHEME In vehicular cloud systems, local data storaes in roadside wireless APs are essential resources for expeditin data dissemination from a data center to mobile vehicles. If more wireless APs that are located on the routes of a vehicle have data chunks, it is more likely that the vehicle can successfully download the data chunks. On the other hand, if too excessive data chunks are transferred to wireless APs, it is a resource waste of local data storae and may cause an increase in network delay from a data center to wireless APs. In this paper, we consider an optimization stratey that maximizes data dissemination success probability when the data storae size of each local data storae is limited. The data dissemination problem is formulated as a binary maximization over X {,1} w u. The objective of the data dissemination problem described in this paper is to maximize the dissemination success probability in the VCS. Due to the stochastic characteristics of θ in P, it is not possible to perfectly uarantee data chunk delivery. Instead, we maximize the dissemination success probability under the assumption that there exists a maximum boundary or quantity of data storae in roadside wireless APs. Let b i denote the maximum number of data chunks that can be stored at the local data storae of the i-th AP. Note that b i is also bounded by the maximum number of data chunks that can be downloaded from a cloud server durin each decision round. Under the assumption that all APs have the same storae capacity,b i is set to b for all i = {1,,w}. Then, we impose a constraint on the selection of the binary decision matrix such that the feasible candidates are from a finite set F or, which

5 is represented as follows: F or = {X : X {,1} w u, X 1 u max b}, 2 where 1 l R l is an all-ones column vector. Let X denote the areate dissemination success probability for all the vehicles and data chunks. Then, X is iven by u X = 1 p i,j. 3 We also define a set function a A X for X, i.e., a A X = X where A X = {i,j : x i,j = 1,1 i w,1 j u} is the index set for x i,j = 1 in X. The optimal binary decision matrix X that maximizes X is to be selected from F or as follows: X = ar maxx. 4 X F or A. Deterministic reedy data dissemination for VCS For simplicity, we first assume that the stochastic characteristics of the network connectivity success rate θ between APs and vehicles are completely observed, thereby bein able to exploit the deterministic statistics for data dissemination. In other words, the values θ = [θ 1,,θ w ] are completely known to the data center in advance. Then, the data dissemination problem is a binary optimization problem, and its optimal solution can be obtained by a deterministic combinatorial alorithm. One may attempt to solve this binary optimization by a brute-search alorithm, which enumerates all possible candidates and checks whether each candidate satisfies the problem statement. However, the complexity rows exponentially with respect to the dimension of X, and thus it is not scalable and limited in applicability to small vehicular systems with only a few tens of w u. To make this binary optimization problem more tractable, we derive the followin proposition and then propose a reedy alorithm based on the proposition. Proposition 1: For maximizin data dissemination success probability, a finite set of binary decision matrices can be reduced as follows: F = {X : X {,1} w u,x 1 u = b 1 w }. Proof: Suppose that an element x y,z X is. When x y,z is chaned from to 1, the variation of the cost function X in 3 with respect to x y,z is v i=1 r i,z w\y 1 s i,k θ k x k,z s i,y θ y, which is reater than or equal to. This implies that as more elements in X chane their value to 1, the variation of X is reater than or equal to the equality holds when r i,z or s i,y is equal to for all i {1,,v}. This means that the cost function X increases over X. Therefore, the cost function in 4 can be maximized with the larest number of 1 s in X that satisfies X 1 u = b 1 w in the constraint. Based on the above proposition, we propose a reedy alorithm that iteratively finds the sub-optimal solution on F by settin one element of X to 1 at each iteration. The detailed procedure is iven in Alorithm 1. The alorithm starts with X, which is one that ives the larest dissemination success Alorithm 1 Deterministic reedy data dissemination for VCS 1: // Initialization 2: X = ar max X {X:X {,1} w u, A X =3}X 3: A X C = φ; 4: // Main loop 5: for l = 4 to w b do 6: {y,z} = ar max {i,j} A1w u \A X A X C 7: a A X A xi,j a A X ; 8: if X 1 u y >= b then 9: A X C = A X C A xy,z ; 1: Go back to line 6. 11: end if 12: A X = A X A xy,z ; 13: X y,z = 1; 14: end for probability amon all the feasible X s with A X = 3 on line 2. 1 Then, at each iteration, the alorithm picks one element x y,z A 1w u \A X A X C maximizin the increment of dissemination success probability, i.e., a A X A xy,z a A X where 1 w u is an all-ones w u matrix, and A X C is the index set for all x i,j s in X that are but are prohibited from bein converted to 1 due to the limited storae capacity of the roadside wireless APs. On line 7, if the number of ones in the y-th row of X is less than b i.e., X 1 u y < b where a y represents the y-th element of vector a, then we set the x y,z in X to 1 i.e., A X = A X A xy,z. Otherwise, we set x y,z in X to i.e., A X C = A X C A xy,z and repeat the above steps. This alorithm terminates when the number of ones in each row of X is equal to b. In Alorithm 1, the number of iterations is equal to w b 3 because each local data storae in the wireless AP has b number of 1 s in its correspondin X s row. In each iteration inside the loop, the maximum from the vector ofw u needs to be searched. The evaluation of a incurs the complexity of Ov u from 3. As a result, the computational complexity of Alorithm 1 becomes Ow 2 u 2 bv, which corresponds to quadratic complexity with respect to w and u. Even thouh we assume that all the wireless APs have the same data storae capacity b in 2, Alorithm 1 can be easily extended to the case in which the capacities are not the same without any sinificant modification. Because the storae capacity of the i-th AP is b i, the number of iterations for findin the optimal solution is set to w i=1 b i 3. Note that the local data storae of the i-th AP has b i number of 1 s in its correspondin X s row. As reedy alorithms may fail to find the lobally optimal solution, it is necessary to check the worst-case performance of a reedy alorithm by checkin its approximation factor. If its approximation is bounded by a constant factor, the reedy alorithm is capable of findin the sub-optimal solution in polynomial-time. To derive its approximation, we derive the followin proposition. 1 Note that the reason for enumeratin all the feasible X s with A X = 3 in the first iteration is to make it possible to derive an approximation of the proposed reedy alorithm, as described in Appendix A.

6 Proposition 2: The proposed dissemination problem in 4 can be transformed into a submodular maximization problem SMP. Proof: Submodularity is an intuitive notion of diminishin returns, which implies that addin an element to a small set ives more returns than addin that same element to a larer set [24]. It is defined as follows: A real-valued set function H, defined on subsets of a finite set S, is called submodular if it satisfies HB 1 s HB 1 HB 2 s HB 2 5 for all B 1 B 2 S and for all s S \ B 2. To verify submodularity, we consider the problem in 4 as follows: max aa X subject to X1 u = b 1 w. 6 X F Note that a A X is also a nondecreasin set function because X is a nondecreasin function over X as shown in Proposition 1. Consider A X1 A X2 A 1v u and A xy,z A 1w u \A X2. Then, we have the followin: a A X1 A xy,z a A X1 = r i,z s i,y θ y x y,z i=1 k {k k,z A X1 } 1 s i,k θ k x k,z, where the riht-hand side is reater than or equal to zero. Then, we can show the followin inequalities: { a A X1 A xk,j a A X1 } { a A X2 A xk,j a A X2 } = s i,k θ k x k,j 1 s i,k θ k x k,j i=1 1 k {k k,j A X2 \A X1 } k {k k,j A X1 } 7 1 s i,k θ k x k,j. 8 Accordin to 8, the function a A X is a sub-modular set function, and the problem in 6 is an SMP. Based on the above proposition, the reedy alorithm is able to achieve a constant factor 1 e 1 approximation of the optimal value of 4. The detailed procedure to achieve such an approximation bound is described in Appendix A. However, in practice, it is desirable to deal with the network connectivity success rate θ as a random process because its statistical property is unknown in advance. In such a case, it is not possible to directly determine how many data chunks are prefetched to the local data storaes. In the next subsection, we propose an alternative way to observe and exploit the stochastic characteristics of θ for maximizin the dissemination success probability in an averae sense. B. Online learnin-based data dissemination alorithm In this paper, we adopt the stochastic multi-armed bandit MAB based online learnin framework presented in [25] to solve the data dissemination problem in 4. MABs are widely used to solve combinatorial optimization problems for cost functions with unknown random variables. The MAB framework radually learns the stochastic characteristics of random variables with unknown distribution and then determines an optimal policy to maximize the cost function in an averae sense. The performance of the MAB is evaluated by analyzin the reret, which is defined as the areated difference between the maximum costs iven by a lobally optimal solution and those by the MAB over time. If the reret increases sub-linearly, it implies that the solution of the MAB radually converes to a lobally optimal solution in a certain number of iterations. In this subsection, we propose an MABbased online learnin alorithm for the data dissemination and perform the reret analysis to show that the solution of our proposed alorithm converes to a lobally optimal solution. 1 Policy desin: In our data dissemination problem, the network connectivity success rate θ for roadside wireless APs is a random variable with unknown distribution that chanes over time. Let n be a time index representin a decision period for online learnin iterations and t = t k w 1 denote the random variables representin network connectivity of the APs, where θ k = E[t k ] for all k = {1,,w}. The proposed MAB-based online learnin alorithm measures the mean network connectivity success rate θ at each decision period and finds an optimal solution on F that maximizes a cost function with mean network connectivity success rate. The lobally optimal binary decision matrix X is iven by u X = ar max 1 h i,j,k s i,k,θ k,x k,j. X F 9 The detailed procedure is iven in Alorithm 2. The idea for this alorithm was inspired by alorithm CWF2 in [25], which exploits the information ained from the operation of each action to determine a dependent action and achieves a loarithmically rowin reret. In Alorithm 2, the initial learnin process is performed for each AP, so that at least one data chunk may be downloaded from the AP to vehicles. On lines 3 8, at thep-th iteration, an arbitrary binary decision matrix X F is chosen such that the number of data chunks downloaded from the p-th AP, which is {S T R X}1 u p, is reater than or equal to 1 in order to measure and estimate the initial values of the instantaneous network connectivity success rate θ k and the accumulated mean network connectivity success rate θ k. Subsequently, the selected arm Xn is played, and θ k and θ k are measured and updated. The instantaneous network connectivity success rate θ = [θ 1,,θ w] T is iven by v u θ k = 1 h i,j,k s i,k,t k,x k,j n {S T. 11 R Xn}1 u k The accumulated mean network connectivity success rate θ = [θ 1,,θ w ] T is updated as follows: θ k = θ k m k +θ k {ST R Xn}1 u k m k +{S T R Xn}1 u k, k = {1,,w}, 12 where m = [m 1,,m w ] T is the number of observation times up to the current iteration for the APs. Based on θ, an optimal binary decision matrix is determined as described in 1 on line 12. The proposed online learnin alorithm

7 Alorithm 2 Proposed online learnin alorithm 1: // Initialization 2: n = ; 3: for p = 1 to w do 4: n := n+1; 5: Play any arm X F such that {S T R X}1 u p 1; 6: θ k = v u ri,j 1 h i,j,ks i,k,t k,x k,j n, k = {1,,w} {S T R Xn}1 u k 7: θ k = θ k m k +θ k {ST R Xn}1 u k m k +{S T R Xn}1 u k, m k = m k +{S T R Xn}1 u k, k = {1,,w}; 8: end for 9: // Main loop 1: while 1 do 11: n := n+1; 12: Play any arm X F which solves the followin maximization problem: u 2 h i,j,k s i,k,θ k,x k,j max{h i,j,k s i,k, max X F 13: θ k = v u ri,j 1 h i,j,ks i,k,t k,x k,j n {S T R Xn}1 u k, k = {1,,w} 14: θ k = θ k m k +θ k {ST R Xn}1 u k m k +{S T R Xn}1 u k, m k = m k +{S T R Xn}1 u k, k = {1,,w}; 15: end while w +1lnn,x k,j,} ; 1 m k iteratively finds a lobally optimal binary decision matrix that maximizes the areate dissemination success probability in an averae sense. Note that θ k radually converes to the actual mean network connectivity success rate asθ k is updated over time. The proposed online learnin alorithm needs two storae units of size w 1 to store θ and the number of observation times m. 2 Reret analysis: We perform the reret analysis to show that the solution of the proposed online learnin alorithm converes to a lobally optimal solution in a certain number of iterations. The reret of the proposed alorithm is an areate discrepancy between the maximum areate dissemination success probabilities by a lobally optimal solution and by the proposed alorithm in Alorithm 2. The reret after N iterations is iven by RN = N X N Xn, 13 n=1 where X = max X F X is the maximum areate dissemination success probability by the optimal binary decision matrix X. The reret analysis derives the upper bound of the reret after N iterations. The upper bound can be obtained as a function of the upper bound of the number of times for which a non-optimal binary decision matrix is selected. Let T NO N denote the number of times for which a non-optimal binary decision matrix is selected for the first N iterations. To show the upper bound of T NO N, we define T k N as a counter for the k-th wireless AP. Once the online learnin alorithm selects a non-optimal binary decision matrix, the index j such that j = ar min k {1,,w} m k is selected, and the correspondin countert j N is increased by1. If there are more than two indexes, one index is selected arbitrarily and the correspondin counter is increased by 1. Then, only one counter will increment its value when the non-optimal binary decision matrix is selected, and the followin equation must hold: w T NO N = T k N. Based on the above equation, the upper bound of reret is iven by w RN max T k N, 14 where max = X min X F X. Under the above inequality, an upper bound of the reret function RN is determined by the upper bound of the counter T k N, which is iven as follows: E[T k N] w+1lnn θ 2 min +1+ π2 3 w2 b, 15 where θ min is a constant less than or equal to 1. The detailed derivation of the upper bound for the counter T k N is described in Appendix B. The upper bound of the reret is as follows: ww +1lnN RN max +w + π2 3 w3 b. 16 θ 2 min As shown in the above equation, the reret function RN increases sub-linearly with respect to the number of iterations. This sub-linear increase implies that the optimal solution of the proposed alorithm radually converes to a lobally optimal binary decision matrix after a certain number of iterations. 3 Comparison with other learnin alorithms: While the proposed alorithm was inspired by the CWF2 alorithm in [25], there are two major enhancements in our proposed alorithm. First, the cost function of the optimization problem in 1 desined for findin the optimal data dissemination

8 1.8 Rayleih fadin, v=26 Rayleih fadin, v=13 Shadowin α=2, v=26 Shadowin α=4, v=26 Shadowin α=4, v=13 probability.6.4.2.1.2.3.4.5.6.7.8.9 1 network connectivity Fi. 2. Map of the roadside wireless APs randomly deployed at the intersections of roads in Beijin. Fi. 3. Simulation results of network connectivity for different propaation models and vehicular subscriber densities. stratey is too complicated to be solved by the CWF2 alorithm. This is because the CWF2 alorithm deals with a cost function that consists of a weihted linear combination where each term is a function of a sinle random variable. On the other hand, our alorithm deals with a more complicated cost function with nonlinear dependencies that include the multiplication of unknown random variables and linear reward terms. Other online learnin alorithms such as UCB1 [26] and LLR [27] are limitedly applicable when the cost function is a function of one random variable and is a weihted linear combination of random variables, respectively. Second, in the CWF2 alorithm, each arm is associated with one unknown random variable, and thus each unknown random variable can be explored only one time in each decision period when the online learnin alorithm decides to exploit the correspondin arm. In contrast, our online learnin alorithm can explore multiple unknown random variables several times in one decision period because each unknown random variable is associated with multiple arms. As a result, our online learnin alorithm finds the features of unknown random variables more rapidly than the CWF2 alorithm. V. PERFORMANCE EVALUATION A. Experiment Environment In this section, we present the results of real-life vehicletrace based experiments desined to evaluate the efficiency of the proposed data dissemination methods for VCSs. It is assumed that the roadside wireless APs are deployed on the intersections of roads, and the vehicular subscribers drive alon routes, which are uided by naviation system and are provided to the cloud service center when the vehicular subscribers request their cloud services. The vehicular subscribers can communicate with the APs located nearby at the intersections on their route. For vehicle traffic trace, we use GPS traces of 2,6 taxis in Beijin as done in [28]. 2 We randomly deploy 4 roadside wireless APs at the intersections as shown in Fiure 2. The vehicular route matrix S is obtained from the route of vehicular subscribers wheres i,j = 1 if the i-th vehicular subscriber oes throuh the intersection at which the j-th AP is deployed. For data dissemination, it is assumed that the data chunks have the same size and the data rates of the APs are identical. The APs access the wireless channel usin IEEE 82.11 DCF. The network connectivity success rate θ = [θ 1,,θ w ] has a positive value in the rane of [, 1]. The chunk request matrix R is randomly set to either or 1 under the assumption that the vehicular subscribers randomly request data chunks. We used the ns-2 network simulator to characterize the network connectivity distribution of the wireless APs. The wireless channels of the links from wireless APs to vehicular subscribers are modeled as either a Rayleih fadin or shadowin model with a path-loss exponent. The path-loss exponent for the wireless channel varies from 2 to 4. Fiure 3 shows the simulation results for network connectivity of the 4 wireless APs for different wireless link models and vehicular subscriber density. The cumulative density function CDF in Fiure 3 indicates that the network connectivity depends heavily on both wireless link characteristics and the number of vehicles passin by the APs. For each wireless link model, the network connectivity becomes worse when the number of vehicles increases. In the shadowin model, the network connectivity becomes worse when the path-loss exponent increases because the communication rane of APs decreases due to the sinificant sinal attenuation. B. Experiment Results for Deterministic Greedy Data Dissemination for VCS In this subsection, we focus on the deterministic reedy data dissemination alorithm and numerically evaluate its performance in various data dissemination scenarios. Each 2 This vehicle-trace dataset was obtained from research conducted by the University of Southern California s Autonomous Networks Research Group, http://anr.usc.edu.

9 mean dissemination success probability 1.8.6.4.2 u = 5 u = 1 u = 2 5 1 15 2 25 3 35 4 number of roadside wireless APs mean dissemination success probability 1.8.6.4.2 u = 1 u = 15 u = 2 1 2 3 4 5 6 7 8 size of local data storae b Fi. 4. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the number of roadside wireless APs Rayleih fadin, v = 26. mean dissemination success probability 1.8.6.4.2 5 1 15 2 25 3 35 4 45 5 number of chunks b = 3 b = 5 b = 1 Fi. 5. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the number of data chunks Rayleih fadin, v = 26. Fi. 6. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the size of local data storae Rayleih fadin, v = 26. mean dissemination success probability 1.8.6.4 Proposed method u=1 Simple cachin method [18]u=1 Proposed method u=15.2 Simple cachin method [18]u=15 Proposed method u=2 Simple cachin method [18]u=2 1 2 3 4 5 6 7 8 size of local data storae b Fi. 7. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the storae size for the proposed and the simple cachin methods Rayleih fadin, v = 26. point in Fiures 4-9 is the averae value of 1 runs for different R s. Fiure 4 depicts the mean dissemination success probability for different numbers of data chunks with respect to the number of roadside wireless APs, for the case when the roadside wireless APs are randomly chosen from the 4 APs depicted in Fiure 2 and the size of the local data storae is set to 3 b = 3. In Fiure 4, we observe that as more APs are deployed on the intersections of the road, the performance of the data dissemination alorithm is enhanced. This is because the vehicular subscribers have more opportunities to download the requested data chunks due to the increasin number of APs located on the route of the vehicular subscribers. This result implies that the proposed alorithm will continue to achieve better data dissemination performance by employin more roadside wireless APs for the VCS. Fiure 5 depicts the averae value of dissemination success probability for different sizes of data storae with respect to the number of data chunks served by the vehicular cloud servers. It shows that the data dissemination performance is deraded when the number of data chunks served by the cloud servers increases. The reason for this deterioration is that the vehicles request more chunks that are diverse while the capacity of local storae at each AP is limited. In order to improve data dissemination performance, the size of the local data storae needs to be increased, and the number of data chunks served by the VCS needs to be decreased. Fiure 6 depicts the averae probability of dissemination success with respect to the size of local data storae. In Fiure 6, the mean probability of dissemination success increases as the size of local data storae increases. This is because the roadside wireless APs can store more data chunks requested by the vehicular subscribers as the size of local data storae increases. However, the increase of data storae size incurs additional costs for the VCS infrastructure. It implies that there is trade-off between the infrastructure expenditure for data storae capacity and the data dissemination performance in VCS.

1 mean dissemination success probability 1.8.6.4.2 Rayleih fadin, v=26 Rayleih fadin, v=13 Shadowin α=2, v=26 Shadowin α=4, v=26 Shadowin α=4, v=13 1 2 3 4 5 6 7 8 size of local data storae mean dissemination success probability.8.7.6.5.4 Greedy alorithm w=5.3 Brute search w=5 Greedy alorithm w=6.2 Brute search w=6 Greedy alorithm w=7 Brute search w=7.1 1 1.5 2 2.5 3 3.5 4 4.5 5 size of local data storae b Fi. 8. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the storae size for different wireless channel models and vehicular subscriber densities u = 2. Fi. 9. Mean dissemination success probability of all vehicle-data chunk pairs with respect to the data storae size for the proposed reedy and the brute search alorithms Rayleih fadin, v = 26, u = 6. For comparin the proposed data dissemination solution with web cachin/proxyin methods, we implemented a simple cachin method. In eneral, most cachin/proxy alorithms have a cachin policy to make each proxy server keep the most popular contents so as to maximize the averae hit rate of the contents requested by the users. The simple replica placement method was implemented to allow each AP to identify and keep the b chunks with the larest number of requests at each AP. In Fiure 7, as the capacity of data storae for roadside wireless APs increases, both dissemination schemes achieve better dissemination performance. In the entire rane, the proposed dissemination scheme outperforms the simple cachin method. Under the simple cachin method, each AP knows the accurate chunk request information of vehicular subscribers, but it does not consider whether its neihborin APs have the same copy of the chunks in their buffers. Furthermore, the simple cachin method does not reflect the different network connectivity statistics for the cachin decision. This experiment result indicates that it is important to exploit the information about the prefetchin of data chunks in neihborin APs and the non-identical wireless connectivity statistics in the data dissemination decision. We next evaluated data dissemination performance with respect to the storae size for the wireless channel models and vehicular subscriber densities. In Fiure 8, the dissemination success probability is hiher when the number of vehicular subscribers v is smaller for both two channel models. In addition, for the shadowin model, the dissemination performance is better when the path-loss exponent α is smaller. The reason is that, as shown in Fiure 3, the network connectivity increases when v and α decrease. This implies that if the network connectivity between vehicular subscribers and roadside wireless APs increases, the dissemination method achieves better dissemination performance. Finally, we verified the sub-optimality of the proposed reedy data prefetchin alorithm. We compared the dissemination success probabilities of the reedy based prefetchin alorithm with those of a brute search alorithm. Note that the brute search alorithm enumerates all the possible data prefetchin cases and then chooses the optimal data prefetchin decision that ives maximum data dissemination performance. Fiure 9 depicts the averae data dissemination success probabilities of the proposed reedy alorithm and brute search alorithm with respect to the data storae size, for the case in which the number of data chunks served by the cloud servers is 6. Fiure 9 shows that the dissemination performance of the proposed reedy alorithm is almost the same as that of the brute search alorithm. This result implies that the proposed reedy alorithm near-optimally distributes the data chunks to the local storae of the APs with reasonable computational complexity as compared to the brute search alorithm. C. Experiment Results for Online Learnin-Based Data Dissemination In this subsection, we focus on the proposed online learninbased data dissemination alorithm and numerically evaluate its performance in a VCS. For this experiment, we set the number of the data chunks served by the cloud servers to 5, and the data storae size is set to 3 u = 5, b = 3. The proposed online learnin alorithm iteratively estimates the network connectivity success rate θ. For the performance evaluation, we compare the reret value of the proposed alorithm with that of a naive online learnin approach usin the UCB1 policy in [26]. The naive online learnin approach is desined to learn the areate data dissemination success probability for all possible data prefetchin sets. Note that the proposed alorithm learns the network connectivity characteristics themselves rather than the areated data dissemination probabilities. Table II represents the data storae size required to store the information learned and updated by the alorithms. Under the UCB1 policy, the data prefetchin set that maximizes Y k + 2lnn/m k is selected at each decision period for prefetchin data chunks to the APs, where n is the number of decision periods, Y k

11 TABLE II SIZE OF DATA STORAGES TO STORE THE INFORMATION LEARNED BY THE PROPOSED ALGORITHM AND THE NAIVE APPROACH UCB1. reret / n Parameters Proposed alorithm Naive approach w = 5,u = 5,b = 3 5 1.24 1 8 w = 6,u = 5,b = 3 6 4.96 1 9 w = 6,u = 1,b = 3 6 8.21 1 56 35 3 25 2 15 1 5 Naive online approach Proposed alorithm -5.5 1 1.5 2 2.5 3 3.5 4 number of iterations n x 1 4 Fi. 1. Reret divided by the number of iterations for static S and R represents the mean observed areate data dissemination success probability of the k-th data prefetchin set, and m k is the number of times that the k-th data prefetchin set has been selected as the decision. As shown in Table II, the w 1 naive approach requires two data storaes of size to u!u b! b! store Y k and m k while the proposed alorithm requires two w 1 data storaes. Therefore, the proposed online learnin alorithm is more efficient than the naive approach in terms of memory resource usae. Fiure 1 depicts the simulation results of the reret divided by the number of iterations, where the vehicle route matrix reret / n 4 35 3 25 2 15 1 5 Naive online approach Proposed alorithm -5.5 1 1.5 2 2.5 3 3.5 4 x 1 4 number of iterations n Fi. 11. Reret divided by the number of iterations where R is abruptly chaned at the 2,-th iteration. accumulated mean network probability 1.8.6.4 AP #1 θ 1 =.6 AP #2 θ.2 2 =.85 AP #3 θ 3 =.78 AP #4 θ 4 =.48 AP #5 θ 5 =.69 5 1 15 2 25 3 35 4 45 5 number of iterations Fi. 12. Estimated network connectivity success rate of each roadside wireless AP with respect to iteration quantity. S and the chunk request matrix R are unchaned durin the simulation. Fiure 1 shows that the reret of the proposed alorithm divided by the number of iterations converes to zero very rapidly in comparison with UCB1. It implies that the proposed online learnin alorithm achieves better data dissemination performance than UCB1 in terms of converence speed. Because the search space for UCB1 includes all possible data prefetchin sets, UCB1 incurs a tremendous time cost to find the optimal data prefetchin decision. Fiure 11 depicts the reret divided by the number of iterations in a dynamically chanin scenario where R chanes at the 2,-th iteration. As shown in Fiure 11, the reret of the proposed alorithm converes consistently while that of the naive approach starts to increase after the 2,-th iteration due to the chane of R. This arises because UCB1 needs to re-learn all the areate dissemination success probabilities when the chunk requests chane. Fiure 12 depicts the estimated network connectivity success rate of each AP durin the iterations of the proposed online learnin alorithm for the scenario in which R is chaned for every iteration. Fiure 12 shows that the estimated network connectivity success rate of the APs rapidly converes to their mean values even thouh R chanes over time. This result implies that the proposed online learnin alorithm is capable of findin the unknown network connectivity success rate of each AP in dynamically chanin vehicular cloud service scenarios. The experiment results with real-life vehicletraces and realistic wireless models indicate that the proposed dissemination method is applicable to data dissemination for VCSs. VI. CONCLUSION In this paper, we have presented a prefetch-based data dissemination method for VCSs consistin of roadside wireless APs with local data storaes. We formulated the data dissemination problem as a combinatorial optimization that maximizes the areate data dissemination success probability when the size of local data storae is limited. Under the assumption

12 that the routes and data request information of vehicle subscribers are readily available, we devised two alorithms to determine how to prefetch a set of data from a data center to roadside wireless APs. The first is a reedy alorithm that solves the dissemination problem when wireless network connectivity characteristics are known deterministically. We proved that this alorithm could find a sub-optimal solution in a polynomial-time by derivin the approximation bound of the reedy alorithm. The second one is an MAB based online learnin alorithm that radually learns the unknown network connectivity success rate at each iteration and then determines an optimal binary decision matrix. In addition, we proved that its optimal solution converes to a lobally optimal solution in a certain number of iterations usin reret analysis. Finally, we presented numerical results of real-life vehicle-trace experiments to demonstrate the performance of the proposed alorithms in a variety of data dissemination scenarios in VCSs. APPENDIX A APPROXIMATION OF THE GREEDY ALGORITHM In this appendix, we show the worst-case performance of the reedy alorithm described in Alorithm 1 by derivin its approximation factor. The problem in 4 can be re-written as the followin optimization problem: max aa X subject to X1 u = b 1 w, 17 X F where a A X = v i=1 u j=1 w 1 s i,k θ k x k,j is a nonneative, nondecreasin, and submodular set function; thus, it satisfies the followin condition: a T a S+ a S {i} a S, 18 i T\S wheret ands are arbitrary sets. Assume that X is an optimal solution of the problem in 17, and X i is a solution of the reedy alorithm after the i-th iteration. We sort the index set of {m,n} i s for X such that a {m,n 1,,m,n i } = max m,n A X \{m,n 1,,m,n i 1} a{m,n 1,, m,n i 1 } {m,n}, i {1,,w b}. 19 Let Z = {m,n 1,m,n 2,m,n 3 } be the set that consists of the first three elements of the index set A X. 3 Then for any element m,n k A X,k 4, and the set Y A 1w u \Z 3 Note that the reedy alorithm enumerates all feasible X {X : X {,1} w u, A X = 3}, and then iteratively finds a sub-optimal solution startin from one of the feasible X s that ives the maximum of the cost function aa X. Thus, the first three elements of the set Z A X are also included in the index set of the reedy solution A X i. {m,n k }, the followin inequalities hold: a Z Y {m,n k } a Z Y a {m,n k } a φ a {m,n 1 } a Z Y {m,n k } a Z Y a {m,n 1 } {m,n k } a {m,n 1 } a {m,n 1 } {m,n 2 } a {m,n 1 } a Z Y {m,n k } a Z Y a {m,n 1 } {m,n 2 } {m,n k } a {m,n 1 } {m,n 2 } a {m,n 1 } {m,n 2 } {m,n 3 } a {m,n 1 } {m,n 2 }. By summin up all above inequalities, we obtain the followin inequality: a Z Y {m,n k } a Z Y 1 3 az. 2 Let i = w b 3 be the total number of iterations of the reedy alorithm and define a new function f a S = a S a Z, which is also a nondecreasin, nonneative, and submodular set function if and only if Z S A 1w u. Accordin to the inequality 18, the followin inequalities also hold: f a A X f a A X i + f a A X i {m,n} f a A X i m,n A X \A X i = f a A X i + a A X i {m,n} a A X i m,n A X \A X i f a A X i +w b 4 ϑ i+1, i {,,i 1}, 21 where ϑ i represents the maximum increment of the set function at the i-th iteration as follows: ϑ i = max m,n A 1w u \A X i 1 aa X i 1 {m,n} A i 1 X c a A X i 1, 22 wherea X i c = {m,n x m,n / A X i,x i 1 1 u m b, m,n} is an index set for the elements of which the value is in X i but at the same time are prohibited from bein included in the index set A X i due to the limited storae capacity of the roadside wireless APs. Thus, we obtain A X i = i j=1 ϑ i for all i {1,,i }. To derive an approximation of the reedy alorithm, we use the inequality in [29]. If P and D are arbitrary positive inteers, ρ i s are arbitrary nonneative reals for i = 1,,P, and ρ 1 >, then P i=1 ρ i min t=1,,p t 1 i=1 ρ i +Dρ t 1 1 1 P D > 1 e P/D. 23

Usin 21 and 23, we obtain the followin inequalities: f a A X i 1 {m,n i } f a A X i j=1 ϑ i k 1 min,,i j=1 ϑ j +w b 4 ϑ k 1 e 1. 24 By combinin 2 and 24, a A X i = a Z+f a A X i a Z+f a A X i 1 {m,n i } f a A X i 1 = a Z+f a A X i 1 a A X i 1 {m,n i } f a A X i 1 {m,n i } a A X i 1 a Z+1 e 1 f a A X 1 3 az = e 1 1 3 az+1 e 1 a A X {m,n i } 1 e 1 a A X. 25 This demonstrates that the reedy alorithm achieves a constant factor approximation bounded by 1 e 1. APPENDIX B UPPER BOUND OF THE COUNTER T k N In this appendix, we derive the upper bound of the counter w+1lnn T k N. Let c n,mk denote m k and I n denote the index of the counter selected at the n-th iteration. Then, the upper bound of the counter T k N can be derived as the followin inequalities: N E[T k N] = 1+ P{I n = k} l+ l+ l+ +1 N n=w+1 N n=w+1 n=w+1 P{I n = k,t k l} { P u 1 h i,j,k s i,k,θ k,x k,j max{h i,j,k s i,k,c n,mk,x k,j,} +1 u 1 h i,j,k s i,k,θ k,x k,j n max{h i,j,k s i,k,c n,mk,x k,j n,}, } T k l N n=w+1 1 { P u min 1<m 1,,m w<t h i,j,k s i,k,θ k,x k,j+1 l+ max{h i,j,k s i,k,c n,mk,x k,j,} u max lm 1,,m w<t 1 +1 h i,j,k s i,k,θ k,x k,j n } max{h i,j,k s i,k,c n,mk,x k,j n,} n=1m 1=1 { u P +1 m w=1m 1=l 1 m w=l h i,j,k s i,k,θ k,x k,j max{h i,j,k s i,k,c n,mk,x k,j,} +1 u 1 h i,j,k s i,k,θ k,x k,j n } max{h i,j,k s i,k,c n,mk,x k,j n,}, 13 26 where l is an arbitrary positive inteer. In order to satisfy the inequality in 26, at least one of the followins must hold: u 1 h i,j,k s i,k,θ k,x k,j X 1 u max{h i,j,k s i,k,c n,mk,x k,j,} 27 u 1 h i,j,k s i,k,θ k,x k,j n Xn+ 1 u max{h i,j,k s i,k,c n,mk,x k,j n,} X < Xn+2 1 u max{h i,j,k s i,k,c n,mk,x k,j n,}. 28 29

14 We next derive the upper bound of the probabilities for 27, 28 and 29. The upper bound for 27 is iven by { u P 1 h i,j,k s i,k,θ k,x k,j X u 1 u P { 1 } max{h i,j,k s i,k,c n,mk,x k,j,} max{h i,j,k s i,k,θ k +c n,mk, w x k,j,} 1 } h i,j,k s i,k,θ k,x k,j u w P{θ k +c n,mk θ k }+P{1 θ k } u w P{θ k +c n,mk θ k }. 3 We use the Chernoff-Hoeffdin bound in 3. Let X 1,,X n be random variables in the rane [,1] such that E[X t X 1,,X t 1 ] = µ, and S n = X 1 + +X n. Then, for all a P{S n nµ+a} e 2a/n and P{S n nµ a} e 2a/n. 31 By the above inequality, the upper bound of the probability P{θ k +c n,mk θ k } for all k = {1,,w} is iven by w+1 lnn P{θ k +c n,mk θ k } e 2m2 k m 2 1 k m k = e 2w+1lnn = n 2w+1. 32 Similarly, the upper bound of the probability for 28 is also iven by w+1 lnn P{θ k θ k +c n,mk } e 2m2 k m 2 1 k m k = e 2w+1lnn = n 2w+1. 33 Last, we consider the last inequality in 29. Because the dissemination failure probability p i,j θ is a decreasin function with respect to θ for all i and j, there exists θ ij for the i-j pair such that the followin inequality holds: p i,j = h i,j,k s i,k,θ ij k,x k,j 1 δ min 2wb, 34 where δ min = X max X F\X X. By 34, the followin inequalities hold: u 1 h i,j,k s i,k,θ min,x k,j,x k,j u p i,j θ ij δ min 2, 35 whereθ min = min i,j,k {i,j,k 1iv,1ju,1kw} θ ij k. If we choose the inteer l w+1lnn in 26, the inequality θ 2 min in 29 does not hold: X Xn 2 1 u max{h i,j,k s i,k,c n,mk,x k,j n,} = X Xn 2 1 u max{h i,j,k s i,k, X Xn 2 u w +1lnN,x k,j n,} m k w +1lnN 1 max{h i,j,k s i,k,,x k,j n,} l u X Xn 2 1 max{h i,j,k s i,k,θ min,x k,j n,} X Xn δ min. Therefore, we do not need to consider the last case in 29. Based on the above inequalities, the upper bound of the counter E[T k N] is iven as follows: E[T k N] l+ n=2m 1=1 m w=1m 1=l m w=l { u w } P{θ k +c n,mk θ k }+P{θ k θ k +c n,mk } w+1lnn θ 2 min + w 2 bn 2w+1 w +1lnN θ 2 min w +1lnN θ 2 min n=2m 1=1 +1+w 2 b n=1 m w=1m 1=l 2n 2 m w=l +1+ π2 3 w2 b. 36 REFERENCES [1] H. Hartenstein and K. P. Laberteaux, A tutorial survey on vehicular ad hoc networks, in IEEE Communications Maazine, vol. 46, no. 6, pp. 164-171, 28. [2] M. J. Khabbaz, C. M. Assi, and W. F. Fawaz, Disruption-tolerant networkin: a comprehensive survey on recent developments and persistin challenes, in IEEE Communications Surveys and Tutorials, vol. 14, no. 2, pp. 67-64, 212. [3] M. Whaiduzzaman, M. Sookhak, A. Gani, and R. Buyya, A survey on vehicular cloud computin, in Journal of Network and Computer Applications, vol. 4, no. 1, pp. 325-344, 214. [4] P. Bellavista, A. Corradi, M. Fanelli, and L. Foschini, A survey of context data distribution for mobile ubiquitous systems, in ACM Computin Surveys, vol. 44, no. 4, pp. 1-49, 213. 2

15 [5] J. Zhao, Y. Zhan, and G. Cao, Data pourin and bufferin on the road: a new data dissemination paradim for vehicular ad hoc networks, in IEEE Transactions on Vehicular Technoloy, vol. 56, no. 6, pp. 3266-3277, 27. [6] B. B. Dubey, N. Chauhan, and S. Pant, RRDD: reliable route based data dissemination technique in vanets, in Proceedins of the 211 International Conference on Communication Systems and Network Technoloies CSNT, 211. [7] X. Bai, M. Gon, Z. Gao, and S. Li, Reliable and efficient data dissemination protocol in vanets, in Proceedins of the IEEE 8th International Conference on Wireless Communications, Networkin and Mobile Computin WiCOM, 212. [8] R. S. Schwarts, A. E. Ohazulike, C. Sommer, H. Scholten, F. Dressler, and P. Havina, Fair and adaptive data dissemination for traffic information systems, in Proceedins of the 212 IEEE Vehicular Networkin Conference VCN, 212. [9] F. Ye, S. Roy, and H. Wan, Efficient data dissemination in vehicular ad hoc networks, in IEEE Journal on Selected Areas in Communications, vol. 3, no. 4, pp. 769-779, 212. [1] M. Sathiamoorthy, A. G. Dimakis, B. Krishnamachari, and F. Bai, Distributed storae codes reduce latency in vehicular networks, in Proceedins of the IEEE International Conference on Computer Communications INFOCOM, 212. [11] H. Lian and W. Zhuan, Cooperative data dissemination via roadside WLANs, in IEEE Communications Maazine, vol. 5, no. 4, pp. 68-74, 212. [12] M. A. Salkuyeh, F. Hendessi, and T. A. Gulliver, Data dissemination with rateless codin in a rid vehicular topoloy, in EURASIP Journal on Wireless Communications and Networkin, vol. 213, no. 1, pp. 1-14, 213. [13] Y. Din and L. Xiao, SAVD: static-node-assisted adaptive data dissemination in vehicular networks, in IEEE Transactions on Vehicular Technoloy, vol. 59, no. 5, pp. 2445-2455, 21. [14] J. Ott and D. Kutscher, Drive-thru internet: IEEE 82.11b for automobile users, in Proceedins of the Twenty-third Annual Joint Conference of the IEEE Computer and Communications Societies INFOCOM, 24. [15] V. Bychkovsky, B. Hull, A. Miu, H. Balakrishnan, and S. Madden, A measurement study of vehicular internet access usin in situ wifi networks, in Proceedins of the ACM 12th Annual International Conference on Mobile Computin and Networkin MobiCom, 26. [16] J. Eriksson, H. Balakrishnan, and S. Madden, Cabernet: vehicular content delivery usin wifi, in Proceedins of the ACM 14th Annual International Conference on Mobile Computin and Networkin Mobi- Com, 28. [17] J. Jeon, S. Guo, Y. Gu, T. He, and D. H. Du, Trajectory-based statistical forwardin for multihop infrastructure-to-vehicle data delivery, in IEEE Transactions on Mobile Computin, vol. 11, no. 1, pp. 1523-1537, 212. [18] J. Wan, A survey of web cachin schemes for the internet, in ACM SIGCOMM Computer Communication Review, vol. 29, no. 5, pp. 36-46, 1999. [19] B. Li, M. J. Golin, G. F. Italiano, and X. Den, On the optimal placement of web proxies in the internet, in Proceedins of the IEEE International Conference on Computer Communications INFOCOM, 1999. [2] L. Qui, V. J. Padmanabhan, and G. M. Voelker, On the placement of web server replicas, in Proceedins of the IEEE International Conference on Computer Communications INFOCOM, 21. [21] A. Nimkar, C. Mandal, and C. Reade, Video placement and disk load balancin alorithm for VoD proxy server, in Proceedins of the IEEE International Conference on Internet Multimedia Services Architecture and Applications IMSAA, 29. [22] I. Giuriu, C. Castillo, A. Tantawi, and M. Steinder, Enablin efficient placement of virtual infrastructures in the cloud, in Proceedins of the International Middleware Conference Middleware, 212. [23] Y. Gao, H. Guan, Z. Qi, Y. Hou, and L. Liu, A multi-objective ant colony system alorithm for virtual machine placement in cloud computin, in Journal of Computer and System Sciences, vol. 79, no. 8, pp. 123-1242, 213. [24] K. Son, H. Kim, Y. Yi, and B. Krishnamachari, Base station operation and user association mechanisms for enery-delay tradeoffs in reen cellular networks, in IEEE Journal on Selected Areas in Communications, vol. 29, no. 8, pp. 1525-1536, 211. [25] Y. Gai and B. Krishnamachari, Online learnin alorithms for stochastic water-fillin, in proceedins of Information Theory and Applications Workshop ITA, 212. [26] P. Auer, N. Cesa-Bianchi, and P. Fischer, Finite-time analysis of the multiarmed bandit problem, in Machine Learnin, vol. 47, no. 2-3, pp. 235-256, 22. [27] Y. Gai, B. Krishnamachari, and R. Jain, Combinatorial network optimization with unknown variables: multi-armed bandits with linear rewards and individual observations, in IEEE/ACM Transactions on Networkin, vol. 2, no. 5, pp. 1466-1478, 212. [28] M. Sathiamoorthy, A. G. Dimakis, B. Krishnamachari, and F. Bai, Distributed storae codes reduce latency in vehicular networks, in Proceedins of the Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies INFOCOM, 212. [29] L. A. Wolsey, Maximizin real-valued submodular functions: primal and dual heuristics for location problems, in Mathematics of Operations Research, vol. 7, no. 3, pp. 41-425, 1982. Ryansoo Kim received his B.S. deree in Information and Communications from Chunnam National University, Daejeon, Republic of Korea, in 21, and his M.S. deree in Information and Communications from the Gwanju Institute of Science and Technoloy GIST, Republic of Korea, in 212. He is currently pursuin a Ph.D. deree in at the School of Information and Communications at GIST. His research interests include location-aware applications in wireless networks and data dissemination in VANET. Hyuk Lim M 3 received his B.S., M.S., and Ph.D. derees all from the School of Electrical Enineerin and Computer Science, Seoul National University SNU, Seoul, Korea in 1996, 1998, and 23, respectively. He is currently an associate professor in the School of Information and Communications, Gwanju Institute of Science and Technoloy GIST, Gwanju, Republic of Korea. He was a postdoctoral research associate in the Department of Computer Science, University of Illinois at Urbana- Champain in 23-26. His research interests include the analytical modelin and empirical evaluation of computer networkin systems, and the network protocol desin and performance analysis for wireless networks. Bhaskar Krishnamachari M 3 received his B.E. in Electrical Enineerin at The Cooper Union, New York, in 1998, and his M.S. and Ph.D. derees from Cornell University in 1999 and 22 respectively. He is currently an Associate Professor and a Min Hsieh Faculty Fellow in the Department of Electrical Enineerin at the University of Southern California s Viterbi School of Enineerin. His primary research interest is in the desin and analysis of alorithms and protocols for next-eneration wireless networks.