DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft

Extrait du fichier (au format texte) :

DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com

Lin Xiao
Microsoft Research AI
Redmond, WA 98052, USA

weiyu@cs.cmu.edu

Adams Wei Yu
Machine Learning Department, Carnegie Mellon University
Pittsburgh, PA 15213, USA

qihang-lin@uiowa.edu

Qihang Lin
Tippie College of Business, The University of Iowa
Iowa City, IA 52245, USA

wzchen@microsoft.com

Weizhu Chen
Microsoft AI and Research
Redmond, WA 98052, USA

October 13, 2017

Abstract
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters
(optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of the model parameters, and update them asynchronously at different machines using local datasets. In this paper, we focus on distributed optimization of large linear models with convex loss functions, and propose a family of randomized primal-dual block coordinate algorithms that are especially suitable for asynchronous distributed implementation with parameter servers. In particular, we work with the saddle-point formulation of such problems which allows simultaneous data and model partitioning, and exploit its structure by doubly stochastic coordinate optimization with variance reduction (DSCOVR). Compared with other first-order distributed algorithms, we show that DSCOVR may require less amount of overall computation and communication, and less or no synchronization. We discuss the implementation details of the DSCOVR algorithms, and present numerical experiments on an industrial distributed computing system.
Keywords: asynchronous distributed optimization, parameter servers, randomized algorithms,
saddle-point problems, primal-dual coordinate algorithms, empirical risk minimization

1. Introduction
Algorithms and systems for distributed optimization are critical for solving large-scale machine learning problems, especially when the dataset cannot fit into the memory or storage of a single machine. In this paper, we consider distributed optimization problems of the form minimize w "R d

Les promotions



A Large-Scale Study of the Evolution of Web Pages - Microsoft
A Large-Scale Study of the Evolution of Web Pages - Microsoft
23/08/2018 - www.microsoft.com
A Large-Scale Study of the Evolution of Web Pages Dennis Fetterly Hewlett Packard Labs 1501 Page Mill Road Palo Alto, CA 94304 dennis.fetterly@hp.com Mark Manasse Marc Najork Microsoft Research Microsoft Research 1065 La Avenida 1065 La Avenida Mountain View, CA 94043 Mountain View, CA 94043 manasse@microsoft.com najork@microsoft.com Janet Wiener Hewlett Packard Labs 1501 Page Mill Road Palo Alto, CA 94304 janet.wiener@hp.com ABSTRACT 1. INTRODUCTION How fast does the web change? Does most...

MSR Quantum applications - Microsoft
MSR Quantum applications - Microsoft
23/08/2018 - www.microsoft.com
( What Can We Do with a Quantum Computer? ( Matthias Troyer  Station Q, ETH Zurich | 1 Classical computers have come a long way Antikythera mechanism ENIAC astronomical positions (1946) (100 BC) Kelvin s harmonic analyzer prediction of tides (1878) Difference Engine (1822) Is there anything that we cannot solve on future supercomputers? Titan, ORNL (2013) Matthias Troyer | | 2 How long will Moore s law continue? Do we see signs of the end of Moore s law? Can we go below 7nm...

DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
23/08/2018 - www.microsoft.com
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com Lin Xiao Microsoft Research AI Redmond, WA 98052, USA weiyu@cs.cmu.edu Adams Wei Yu Machine Learning Department, Carnegie Mellon University Pittsburgh, PA 15213, USA qihang-lin@uiowa.edu Qihang Lin Tippie College of Business, The University of Iowa Iowa City, IA 52245, USA wzchen@microsoft.com Weizhu Chen Microsoft AI and Research Redmond, WA 98052, USA October 13,...

Vers une approche simplifiée pour introduire le caractère ... - Microsoft
Vers une approche simplifiée pour introduire le caractère ... - Microsoft
23/11/2017 - www.microsoft.com
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/262881208 Vers une approche simplifiée pour introduire le caractère incrémental dans les systèmes de dialogue Conference Paper · July 2014 CITATION READS 1 26 3 authors, including: Hatim Khouzaimi Romain Laroche Orange Labs / Laboratoire Informatique d'Avi & Microsoft Maluuba 12 PUBLICATIONS 42 CITATIONS 58 PUBLICATIONS 185 CITATIONS SEE PROFILE All content following this page was uploaded by Hatim Khouzaimi on 28 April 2015. The user has requested enhancement of the downloaded file. SEE PROFILE 21ème...

1 Introduction - Microsoft
1 Introduction - Microsoft
11/04/2018 - www.microsoft.com
One-Way Accumulators: A Decentralized Alternative to Digital Signatures (Extended Abstract) Josh Benaloh Clarkson University Michael de Mare Giordano Automation Abstract This paper describes a simple candidate one-way hash function which satis es a quasi-commutative property that allows it to be used as an accumulator. This property allows protocols to be developed in which the need for a trusted central authority can be eliminated. Space-e cient distributed protocols are given for document time...

Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
22/05/2017 - www.microsoft.com
 ''&'''''' '&'!' &'' &''&''''''' ' ' ''''''''''''"' ''#' '$'%&''&&'''*')'+'!',''-''''.')'+' '/ ')'0''1&''!''2 ''3 '4'6'5'8'7''9';':'=''§'H''£'Œ'X'© '’''“'”'','¾'K''‘''£'Œ'‹'“'”!’'8'’''Š''Œ''Š''›'ž'’'''£'Œ'ž'Š'­'Š',!’'8'’'''£!’'H'¥&`''œ'Š',!”''Š',!’'8'’'''£!’'H'™&'Œ'ž'“'”'¥&`'“'œ'™'H'“'œ'’'¸'¨'£'²'‹'¬''Ž'@'Ž&`'›'ž'Š',''œ'¨$i'›'ž'§'V'Š',''£'®%Ï'“'”!’'H'¥'H'»&`'’'' 'H'Š'­!”''Š'z''£!’'K'“'”!’'H'¥ 'Ž'£'$c'’'' 'H'Š','›'ž'Š$e'’''Š''Œ'!”''›'­'“'”'›'´''£'›'´''¢'Ž&`''œ''”'Ž'h'¤'‡'›','²'>'±''¥&`'Œ''t'§'H' '0'“'”!’'¯'’'' 'H'Œ''Š''Š'#'©'P'™'H'“'”!”''Š',!’'H'›'ž'“'”'Ž&`!’'H''£''¹'›''§'|''£'''Š'¼'“'”'›'Q'Š',!’'8'’'.''£!’'K'¥&`''”'Š''™ '“'«''´'“'«'’'w''z''£!’'H!’'K'Ž'£'’$i'Ÿ'V'Š'0'R'n'§'H'¾'H''”''œ'Š','™'p''£'§'|'t'Œ'ž'’'zÏ&'¤'‡' 'K'Š',!’Ð'’'' 'H'Š''Š','™'H'¥&`'Š''›''t'Œ''Š'+'Œ''Š''¥'8''£'Œ''™'K'Š','™Ñ't'›$i'§'K' %Ï'¨'@'›'ž'“'”''z't''µ'''Ž&`!’%²'© !’'H'Š''''’''“'”'Ž&`!’'H'›$i!”&''£'™'H'Š''Ž'£''*'Š',''”''£'›'ž'’''“'”'t'²'0'±Ò'¥&`'Œ'''£'§'H' Ð'“'”'›$i'Œ''“'”'¥'£'“'”'™'p'“'œ''­'“'«'’$i''z''£!’'H!’'K'Ž'£'’'º'Ÿ'"'Š'1'R'n'™'H'Š'#''¢'Ž&`'Œ'!”''Š','™&Ï$c'¤'‡' 'H'Š'!’ '’'' 'H'Š'º'Š','™'K'¥&`'Š','›'w''£'Œ'ž'Š'º'Œ''Š''¥'8''£'Œ'ž'™'H'Š','™Ó''£'›'-'›''Ž&`''œ'“'”'™''Œ'ž'Ž%²'™'K'›'-'¤'‡' 'H'“'”''.' '…'',''£!’'…'§'H'“'«'¶&`'Ž'£'’'w''F'’'-'’'' 'K'Š''¶&`'Š''Œ'ž'’''“'œ'','Š''›','²$i'·'*' 'H'Š','›'ž'Š '“'œ!’%Ï'’''¾'H'“'œ'’''“'œ'¶'£'Š$e!’'H'Ž'£'’''“'”'Ž&`!’'H'›'­'¤'‡'“'œ''”'&'Ÿ'"'Š$e''¢'Ž&`'Œ'ž!”'¯''£''œ'“'”'›'ž'Š','™'0''‘''F'’''Š','Œ''² Ô'=!’'8'’'.''£!’'H'¥'£''”'Š',!”''Š',!’'8'’'''£!’'H'™'p'Œ''“'œ'¥&`'“'”'™'H'“'«'’'¸'¨&c'“'œ!’'p'§'"'Š','Œ'ž'','Ž&`''”'t'’''“'œ'Ž&`!’'p''£'Œ''Š''Ž'£''´'“'”!’'8'’''Š''Œ''Š','›'X'’'''¢'Ž'£'Œ$i'›'ž'Š''¶'£'Š','Œ''t''Q'Œ'ž'Š'z''F'© '›'ž'Ž&`!’'H'›','²ÖÕ×'“'”'Œ''›'X'’'''«'¨&`'»'*'’'' 'H'Š'#'¨Ø' '|''z'¶&`'Š'p'“'”!”''§'"'Ž&`'Œ'ž'’'''£!’'8'’'...

Architectures reconfigurables et traitement de proble`mes ... - Microsoft
Architectures reconfigurables et traitement de proble`mes ... - Microsoft
16/11/2016 - www.microsoft.com
RECHERCHE Architectures reconfigurables et traitement de proble`mes NP-difficiles : un nouveau domaine d application Youssef Hamadi    David Merceron  '  ' LIRMM, UMR 5506 CNRS/Universite´ Montpellier II 161, Rue Ada, 34392 Montpellier Cedex 5 hamadi@lirmm.fr ''' EURIWARE, 12-14 rue du fort de St-Cyr 78067 St Quentin-en-Yvelines Cedex damercer@euriware.fr RE´SUME´. L algorithme GSAT est un algorithme de recherche locale. Cette me´thode recherche la premie`re instanciation...

MatrixExplorer: Un système pour l'analyse exploratoire de ... - Microsoft
MatrixExplorer: Un système pour l'analyse exploratoire de ... - Microsoft
22/05/2017 - www.microsoft.com
MatrixExplorer: Un système pour l analyse exploratoire de réseaux sociaux Nathalie Henry Jean-Daniel Fekete INRIA Futurs/LRI/University of Sydney Bât 490, Université Paris-Sud 91405 Orsay Cedex Nathalie.Henry@lri.fr INRIA Futurs/LRI Bât 490, Université Paris-Sud 91405 Orsay Cedex Jean-Daniel.Fekete@inria.fr RESUME ABSTRACT Dans cet article, nous présentons le système MatrixExplorer destiné à explorer des réseaux sociaux. Il a été conçu pour des chercheurs en sciences sociales...
 
 

CZ5702485_01-FR_Mise en page 1 - Sauter électroménager
CZ5702485_01-FR_Mise en page 1 - Sauter électroménager
09/02/2018 - www.sauter-electromenager.com
GUIDE D'INSTALLATION ET D UTILISATION Table de cuisson Chère Cliente, Cher Client, Vous venez d'acquérir une table SAUTER et nous vous en remercions. C'est pour vous que nous avons conçu cette nouvelle génération d'appareils pour vous permettre jour après jour d'exercer sans retenue vos talents de chef et votre créativité, pour vous faire plaisir et faire plaisir à vos amis et votre famille. Votre nouvelle table SAUTER s'intègrera harmonieusement dans votre cuisine et alliera parfaitement...

Communiqué de presse (pdf) - Kyocera
Communiqué de presse (pdf) - Kyocera
16/05/2012 - www.kyocera.fr
Nouvelle certification et extension de garantie pour les modules solaires Kyocera Solar : démonstration de l'excellente qualité « Made in Europe » Kyoto / Neuss, 17. Octobre 2011 ­ pour le groupe technologique japonais Kyocera, l'assurance qualité et la transparence jouent un rôle primordial. Depuis peu, le site européen de production du groupe de renommée internationale dans le secteur du photovoltaïque, implanté en République Tchèque, est certifié par l'organisme TÜV Rheinland...

Deskbooking Datasheet
Deskbooking Datasheet
17/09/2024 - www.logitech.com
Datasheet Logitech Essential and Select for Flex Desks Manage shared desk environments with Logitech service plans for flex desks. Logitech Essential for Flex Desks and Select for Flex Desks service plans provide easy desk booking, intuitive maps, advanced management through Logitech Sync, and comprehensive product coverage. These plans ensure a seamless and efficient workspace experience for administrators and employees. Easy desk booking Empower your team with easy desk booking through the Logitech...

Application Form - Samsung
Application Form - Samsung
21/11/2014 - www.samsung.com
2008 SAMSUNG Joint Universities Creative Technology Talents Awards Application Form Eligibility 1. Students from 5 local Universities (including Hong Kong Polytechnic University, The Chinese University of Hong Kong, City University of Hong Kong, The Hong Kong University of Science and Technology and The University of Hong Kong) 2. Teams can only be comprised of members (2 to 4 people) of the same school. 3. An applicant cannot be a member of more than one team. Each team is allowed to submit...

LMX21986 - LG Electronics
LMX21986 - LG Electronics
24/11/2014 - www.lg.com
R E F R I G E RATO RS LMX21986 FRENCH DOOR LMX21986 20.5 Cu.Ft. Total Capacity (Counter Depth Refrigerator) Style and Design " French Door Refrigerator with Slim SpacePlus !" Ice System and Double Freezer Drawers " Counter Depth " Premium Finish in Stainless Steel " Contoured Doors with Matching Commercial Handles and Pocket Freezer Handles " Hidden Hinges " Premium LED Interior Light Organization " 3 Slide-Out, Spill-Protector !" Tempered Glass Shelves/1 Folding Shelf " Slim SpacePlus!"...

WD Sentinel™ DX4000 Small Office Storage Server - Product ...
WD Sentinel™ DX4000 Small Office Storage Server - Product ...
11/04/2012 - www.wdc.com
WD Sentinel DX4000 Connecter. Protéger. Collaborer. TM Serveur de stockage pour PME Installation facile. Facile à gérer. Protection de données complète. Accès à distance intégré sécurisé. WD Sentinel est facile à installer et fournit une solution de stockage sécurisée et centralisée, une protection de données complète et un accès aux données pour vos employés, clients ou partenaires, depuis vos bureaux ou à l'extérieur. Grâce à une technologie matérielle et logicielle...