DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft

Extrait du fichier (au format texte) :

DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com

Lin Xiao
Microsoft Research AI
Redmond, WA 98052, USA

weiyu@cs.cmu.edu

Adams Wei Yu
Machine Learning Department, Carnegie Mellon University
Pittsburgh, PA 15213, USA

qihang-lin@uiowa.edu

Qihang Lin
Tippie College of Business, The University of Iowa
Iowa City, IA 52245, USA

wzchen@microsoft.com

Weizhu Chen
Microsoft AI and Research
Redmond, WA 98052, USA

October 13, 2017

Abstract
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters
(optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of the model parameters, and update them asynchronously at different machines using local datasets. In this paper, we focus on distributed optimization of large linear models with convex loss functions, and propose a family of randomized primal-dual block coordinate algorithms that are especially suitable for asynchronous distributed implementation with parameter servers. In particular, we work with the saddle-point formulation of such problems which allows simultaneous data and model partitioning, and exploit its structure by doubly stochastic coordinate optimization with variance reduction (DSCOVR). Compared with other first-order distributed algorithms, we show that DSCOVR may require less amount of overall computation and communication, and less or no synchronization. We discuss the implementation details of the DSCOVR algorithms, and present numerical experiments on an industrial distributed computing system.
Keywords: asynchronous distributed optimization, parameter servers, randomized algorithms,
saddle-point problems, primal-dual coordinate algorithms, empirical risk minimization

1. Introduction
Algorithms and systems for distributed optimization are critical for solving large-scale machine learning problems, especially when the dataset cannot fit into the memory or storage of a single machine. In this paper, we consider distributed optimization problems of the form minimize w "R d

Les promotions



Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
22/05/2017 - www.microsoft.com
''&'''''' '&'!' &'' &''&''''''' ' ' ''''''''''''"' ''#' '$'%&''&&'''*')'+'!',''-''''.')'+' '/ ')'0''1&''!''2 ''3 '4'6'5'8'7''9';':'='''H''''X' '''''',''K''''''''!'8'''''''''''''''''''',!'8''''!'H'&`'''',!''',!'8''''!'H'&'''''&`''''H''''''''''''@'&`'''','''$i''''V'','''%''!'H''H'&`''' 'H''!'''z''!'K''!'H' '''$c''' 'H'','''

DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
23/08/2018 - www.microsoft.com
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com Lin Xiao Microsoft Research AI Redmond, WA 98052, USA weiyu@cs.cmu.edu Adams Wei Yu Machine Learning Department, Carnegie Mellon University Pittsburgh, PA 15213, USA qihang-lin@uiowa.edu Qihang Lin Tippie College of Business, The University of Iowa Iowa City, IA 52245, USA wzchen@microsoft.com Weizhu Chen Microsoft AI and Research Redmond, WA 98052, USA October 13, 2017 Abs

1 Introduction - Microsoft
1 Introduction - Microsoft
11/04/2018 - www.microsoft.com
One-Way Accumulators: A Decentralized Alternative to Digital Signatures (Extended Abstract) Josh Benaloh Clarkson University Michael de Mare Giordano Automation Abstract This paper describes a simple candidate one-way hash function which satis es a quasi-commutative property that allows it to be used as an accumulator. This property allows protocols to be developed in which the need for a trusted central authority can be eliminated. Space-e cient distributed protocols are given for document time

C dric FOURNET LE JOIN-CALCUL : UN CALCUL POUR ... - Microsoft
C dric FOURNET LE JOIN-CALCUL : UN CALCUL POUR ... - Microsoft
11/04/2018 - www.microsoft.com
TH SE pr sent e L' COLE POLYTECHNIQUE pour obtenir le titre de DOCTEUR DE L' COLE POLYTECHNIQUE sp cialit : INFORMATIQUE par C dric FOURNET Sujet de la th se : LE JOIN-CALCUL : UN CALCUL POUR LA PROGRAMMATION R PARTIE ET MOBILE The Join-Calculus: a Calculus for Distributed Mobile Programming Soutenue le 23 Novembre 1998 devant le jury compos de : MM. Robin Milner Roberto Amadio G rard Boudol Jean-Jacques L vy G rard Berry Luca Cardelli Georges Gonthier Pr sident Rapporteurs Directeur de th se Ex

D6. 4: Final evaluation of CLASSiC TownInfo and ... - Microsoft
D6. 4: Final evaluation of CLASSiC TownInfo and ... - Microsoft
23/11/2017 - www.microsoft.com
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228835240 D6. 4: Final evaluation of CLASSiC TownInfo and Appointment Scheduling systems Article May 2011 CITATIONS READS 15 56 11 authors, including: Helen Hastie Filip Jurcicek Heriot-Watt University Charles University in Prague 105 PUBLICATIONS 858 CITATIONS 55 PUBLICATIONS 439 CITATIONS SEE PROFILE SEE PROFILE Oliver Joseph Lemon Steve Young Heriot-Watt University Univers

A Large-Scale Study of the Evolution of Web Pages - Microsoft
A Large-Scale Study of the Evolution of Web Pages - Microsoft
23/08/2018 - www.microsoft.com
A Large-Scale Study of the Evolution of Web Pages Dennis Fetterly Hewlett Packard Labs 1501 Page Mill Road Palo Alto, CA 94304 dennis.fetterly@hp.com Mark Manasse Marc Najork Microsoft Research Microsoft Research 1065 La Avenida 1065 La Avenida Mountain View, CA 94043 Mountain View, CA 94043 manasse@microsoft.com najork@microsoft.com Janet Wiener Hewlett Packard Labs 1501 Page Mill Road Palo Alto, CA 94304 janet.wiener@hp.com ABSTRACT 1. INTRODUCTION How fast does the web change? Does most of th

User-Driven Access Control: Rethinking Permission ... - CiteSeerX
User-Driven Access Control: Rethinking Permission ... - CiteSeerX
23/08/2018 - www.microsoft.com
User-Driven Access Control: Rethinking Permission Granting in Modern Operating Systems Franziska Roesner, Tadayoshi Kohno {franzi, yoshi}@cs.washington.edu University of Washington Alexander Moshchuk, Bryan Parno, Helen J. Wang {alexmos, parno, helenw}@microsoft.com Microsoft Research, Redmond Crispin Cowan crispin@microsoft.com Microsoft Abstract tionality and security for access to the user s data and resources. From a functionality standpoint, isolation inhibits the client-side manipulation

MSR Quantum applications - Microsoft
MSR Quantum applications - Microsoft
23/08/2018 - www.microsoft.com
( What Can We Do with a Quantum Computer? ( Matthias Troyer  Station Q, ETH Zurich | 1 Classical computers have come a long way Antikythera mechanism ENIAC astronomical positions (1946) (100 BC) Kelvin s harmonic analyzer prediction of tides (1878) Difference Engine (1822) Is there anything that we cannot solve on future supercomputers? Titan, ORNL (2013) Matthias Troyer | | 2 How long will Moore s law continue? Do we see signs of the end of Moore s law? Can we go below 7nm feature size?

Plateforme Intel
Plateforme Intel
07/02/2012 - event.asus.com
ASUS Server Motherboard WHQL Intel DP Platform Product Name P7F-E P7F-M P7F-X P7F-C/SAS P7F-C/4L Z8PE-D12X Z8PE-D18 Z8NA-D6 Z8NR-D12 Z8NA-D6C Z8NH-D12 DSBF-D/SAS DSBF-D DSBF-D12/SAS DSBF-D12 DSBF-D/1U DSBF-DE/SAS DSBF-DE DSBV-D DSBV-D/C DSBV-DX/C DSBV-DX/SAS DSBV-DX DSAN-DX DSEB-DG/SAS DSEB-D16/SAS DSEB-DG DSGC-DW DSGC-DW/SAS 1259027 1284247 1284356 1283335 1299864 1201046 1218745 1219150 1220679 1221171 1226414 1227733 1215876 1215876 1257348 1258323 1259027 1284247 1284356 1283335 1299864 1332

A R K E  O N F I N A N C E
A R K E O N F I N A N C E
16/03/2012 - www.archos.com
ARCHOS Socit anonyme au capital de 10.174.172 euros Sige Social : 12, rue Ampre ZI Igny 91430 IGNY 343 902 821 RCS EVRY NOTE D'OPRATION Mise la disposition du public l'occasion de l'mission et de l'admission sur Euronext Paris d'actions nouvelles, souscrire en numraire, dans le cadre d'une augmentation de capital avec maintien du droit prfrentiel de souscription des actionnaires, d'un montant brut, prime d'mission incluse, de 26 452 842 euros par mission de 4 069 668 actions

Solutions mobiles et sans fil - TomTom
Solutions mobiles et sans fil - TomTom
22/02/2017 - www.tomtom.com
Solutions mobiles et sans fil Dcouvrez comment les solutions gospatiales de TomTom peuvent enrichir la localisation dans vos applications cartes et contenu | trafic temps rel & historique | lbs www.tomtom.com/licensing Solutions mobiles et sans fil Le march des mobiles inclut les appareils de type PND et appareils photo numriques. Le secteur du sans fil comprend les trois modes de distribution en matire de commercialisation de contenu et d'applications auprs des abonns des services mobi

PDF Download - Samsung
PDF Download - Samsung
21/11/2014 - www.samsung.com
Samsung Annual Report 2003 THE NEXT PHASE THE NEXT PHASE Samsung Annual Report 2003 THE NEXT PHASE MOTIVATION To understand where we re going, it helps to take a look at what s driving us there. 9 COLLABORATION Successful companies no longer always go it alone. 23 EXPECTATION Customers are all about expectations. 35 DEMONSTRATION It is important to demonstrate that our standards, values and goals are indeed real. 44 COMPUTATION 61 ORGANIZATION The power of Samsung as a brand is tied to the st

Fiche produit Sony : 48/1237482829948.pdf
Fiche produit Sony : 48/1237482829948.pdf
16/02/2012 - www.sony.fr
Sony Professional PRESS RELEASE New PHU220R boosts continuous shooting for XDCAM users 9 September 2011 AMSTERDAM (IBC Booth 12.A10), 9 September, 2011: Sony Professional Media increases the efficiency of XDCAM even further with the launch of the new PHU-220R. This latest Professional Hard Disk unit offers 220GB of full continuous recording to professional videographers using XDCAM EX* & the latest XDCAM camcorders. Successor to the light and robust PHU-120R, the new PHU-220R enables end-users t

KM40 Manuel d'instruction Bedienungsanleitung ... - H.Koenig
KM40 Manuel d'instruction Bedienungsanleitung ... - H.Koenig
05/01/2017 - www.hkoenig.com
KM40 Manuel d'instruction Bedienungsanleitung Gebruiksaanwijzing Manual de instrucciones Manuale d uso Veillez conserver ce manuel pour vous y rfrer ds que vous en aurez besoin Bitte bewahren Sie dieses Dokument auf. Sie koennten es in Zukunft als Informationsquelle benoetigen. Het handboek bewaren voor toekomstig gebruik. Asegrese de guardar este manual para futuras referencias o cuando lo necesite Assicurarsi di conservare questo manuale per ogni futuro riferimento e necessit FRANCAIS