DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft

Extrait du fichier (au format texte) :

DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com

Lin Xiao
Microsoft Research AI
Redmond, WA 98052, USA

weiyu@cs.cmu.edu

Adams Wei Yu
Machine Learning Department, Carnegie Mellon University
Pittsburgh, PA 15213, USA

qihang-lin@uiowa.edu

Qihang Lin
Tippie College of Business, The University of Iowa
Iowa City, IA 52245, USA

wzchen@microsoft.com

Weizhu Chen
Microsoft AI and Research
Redmond, WA 98052, USA

October 13, 2017

Abstract
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters
(optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of the model parameters, and update them asynchronously at different machines using local datasets. In this paper, we focus on distributed optimization of large linear models with convex loss functions, and propose a family of randomized primal-dual block coordinate algorithms that are especially suitable for asynchronous distributed implementation with parameter servers. In particular, we work with the saddle-point formulation of such problems which allows simultaneous data and model partitioning, and exploit its structure by doubly stochastic coordinate optimization with variance reduction (DSCOVR). Compared with other first-order distributed algorithms, we show that DSCOVR may require less amount of overall computation and communication, and less or no synchronization. We discuss the implementation details of the DSCOVR algorithms, and present numerical experiments on an industrial distributed computing system.
Keywords: asynchronous distributed optimization, parameter servers, randomized algorithms,
saddle-point problems, primal-dual coordinate algorithms, empirical risk minimization

1. Introduction
Algorithms and systems for distributed optimization are critical for solving large-scale machine learning problems, especially when the dataset cannot fit into the memory or storage of a single machine. In this paper, we consider distributed optimization problems of the form minimize w "R d

Les promotions



Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
Entanglement and Rigidity in Percolation Models ... - Alexander Holroyd
22/05/2017 - www.microsoft.com
 ''&'''''' '&'!' &'' &''&''''''' ' ' ''''''''''''"' ''#' '$'%&''&&'''*')'+'!',''-''''.')'+' '/ ')'0''1&''!''2 ''3 '4'6'5'8'7''9';':'=''§'H''£'Œ'X'© '’''“'”'','¾'K''‘''£'Œ'‹'“'”!’'8'’''Š''Œ''Š''›'ž'’'''£'Œ'ž'Š'­'Š',!’'8'’'''£!’'H'¥&`''œ'Š',!”''Š',!’'8'’'''£!’'H'™&'Œ'ž'“'”'¥&`'“'œ'™'H'“'œ'’'¸'¨'£'²'‹'¬''Ž'@'Ž&`'›'ž'Š',''œ'¨

MSR Quantum applications - Microsoft
MSR Quantum applications - Microsoft
23/08/2018 - www.microsoft.com
( What Can We Do with a Quantum Computer? ( Matthias Troyer  Station Q, ETH Zurich | 1 Classical computers have come a long way Antikythera mechanism ENIAC astronomical positions (1946) (100 BC) Kelvin s harmonic analyzer prediction of tides (1878) Difference Engine (1822) Is there anything that we cannot solve on future supercomputers? Titan, ORNL (2013) Matthias Troyer | | 2 How long will Moore s law continue? Do we see signs of the end of Moore s law? Can we go below 7nm feature size?

DictaNum : système de dialogue incrémental pour la dictée ... - Microsoft
DictaNum : système de dialogue incrémental pour la dictée ... - Microsoft
23/11/2017 - www.microsoft.com
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/262881756 DictaNum : système de dialogue incrémental pour la dictée de numéros. Conference Paper · July 2014 CITATIONS READS 0 57 3 authors, including: Hatim Khouzaimi Romain Laroche Orange Labs / Laboratoire Informatique d'Avi & Microsoft Maluuba 12 PUBLICATIONS 42 CITATIONS 58 PUBLICATIONS 185 CITATIONS SEE PROFILE SEE PROFILE All content following this page was uploaded

DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
DSCOVR: Randomized Primal-Dual Block Coordinate ... - Microsoft
23/08/2018 - www.microsoft.com
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization lin.xiao@microsoft.com Lin Xiao Microsoft Research AI Redmond, WA 98052, USA weiyu@cs.cmu.edu Adams Wei Yu Machine Learning Department, Carnegie Mellon University Pittsburgh, PA 15213, USA qihang-lin@uiowa.edu Qihang Lin Tippie College of Business, The University of Iowa Iowa City, IA 52245, USA wzchen@microsoft.com Weizhu Chen Microsoft AI and Research Redmond, WA 98052, USA October 13, 2017 Abs

C dric FOURNET LE JOIN-CALCUL : UN CALCUL POUR ... - Microsoft
C dric FOURNET LE JOIN-CALCUL : UN CALCUL POUR ... - Microsoft
11/04/2018 - www.microsoft.com
TH SE pr sent e L' COLE POLYTECHNIQUE pour obtenir le titre de DOCTEUR DE L' COLE POLYTECHNIQUE sp cialit : INFORMATIQUE par C dric FOURNET Sujet de la th se : LE JOIN-CALCUL : UN CALCUL POUR LA PROGRAMMATION R PARTIE ET MOBILE The Join-Calculus: a Calculus for Distributed Mobile Programming Soutenue le 23 Novembre 1998 devant le jury compos de : MM. Robin Milner Roberto Amadio G rard Boudol Jean-Jacques L vy G rard Berry Luca Cardelli Georges Gonthier Pr sident Rapporteurs Directeur de th se Ex

D6. 4: Final evaluation of CLASSiC TownInfo and ... - Microsoft
D6. 4: Final evaluation of CLASSiC TownInfo and ... - Microsoft
23/11/2017 - www.microsoft.com
See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/228835240 D6. 4: Final evaluation of CLASSiC TownInfo and Appointment Scheduling systems Article · May 2011 CITATIONS READS 15 56 11 authors, including: Helen Hastie Filip Jurcicek Heriot-Watt University Charles University in Prague 105 PUBLICATIONS 858 CITATIONS 55 PUBLICATIONS 439 CITATIONS SEE PROFILE SEE PROFILE Oliver Joseph Lemon Steve Young Heriot-Watt University Univer

User-Driven Access Control: Rethinking Permission ... - CiteSeerX
User-Driven Access Control: Rethinking Permission ... - CiteSeerX
23/08/2018 - www.microsoft.com
User-Driven Access Control: Rethinking Permission Granting in Modern Operating Systems Franziska Roesner, Tadayoshi Kohno {franzi, yoshi}@cs.washington.edu University of Washington Alexander Moshchuk, Bryan Parno, Helen J. Wang {alexmos, parno, helenw}@microsoft.com Microsoft Research, Redmond Crispin Cowan crispin@microsoft.com Microsoft Abstract tionality and security for access to the user s data and resources. From a functionality standpoint, isolation inhibits the client-side manipulation

1 Introduction - Microsoft
1 Introduction - Microsoft
11/04/2018 - www.microsoft.com
One-Way Accumulators: A Decentralized Alternative to Digital Signatures (Extended Abstract) Josh Benaloh Clarkson University Michael de Mare Giordano Automation Abstract This paper describes a simple candidate one-way hash function which satis es a quasi-commutative property that allows it to be used as an accumulator. This property allows protocols to be developed in which the need for a trusted central authority can be eliminated. Space-e cient distributed protocols are given for document time

Voir l'actualité PDF - Heden
Voir l'actualité PDF - Heden
27/10/2016 - www.heden.fr
COMMUNIQUE DE PRESSE Noisy-le-Grand, le 24 décembre 2013 Coques de protection pour Smartphone Samsung GALAXY Note 3 et S4 La marque Heden propose deux nouvelles gammes de protections pour les Smartphones Samsung Galaxy Note 3 et Galaxy S4. Idéal pour protéger les mobiles des rayures, des chocs et de la poussière, ces coques sont légères et durables. Elles permettent l'accès à tous les boutons, les contrôles ainsi que les ports de rechargement. Facile à installer et à retirer, ces coques

Quick-Start Guide Guide de démarrage rapide
Quick-Start Guide Guide de démarrage rapide
16/02/2012 - www.logitech.com
1 2 Headphones overview A. B. C. D. E. F. G. H. Logitech® FreePulse Wireless!" Headphones Power button / Connect button Volume controls Wireless adapter Indicator lights Power adapter/charge connector Reset button Custom fit iPod® plates (6) Présentation des casques A. B. C. D. E. F. G. H. Quick-Start Guide Guide de démarrage rapide 4 Logitech® FreePulse Wireless!" Headphones/Casque www.logitech.com 5 Technical specifications: Logitech® Music Anywhere!" Wireless Specifications. " Range: up

Nouveaux sites de production de Kyocera Solar en République ...
Nouveaux sites de production de Kyocera Solar en République ...
16/05/2012 - www.kyocera.fr
Communiqué de presse Kyocera accroît sa capacité de production solaire Nouveaux sites de production de Kyocera Solar en République tchèque et en Chine Kyoto / Neuss, 21 janvier 2011 ­ Le groupe technologique japonais Kyocera, l'un des fabricants leaders dans le secteur du solaire, augmente ses capacités de production solaire sur deux sites. Sur le site de Kadan, en République tchèque, la production sera élargie à un nouveau bâtiment tandis qu'à Tianjin (Chine), une nouvelle usine va r

MFQ7-IS
MFQ7-IS
18/06/2012 - www.smeg.fr
MFQ7-IS SMEG ELITE Robinet mitigeur finition inox brossé EAN13: 8017709101251 Mono commande Bras rotatif jusqu'à 120° Coordonné à la gamme Quadra SMEG France - 9, rue Linus Carl Pauling - B.P. 548 - 76824 MONT SAINT AIGNAN Cedex - Tél.: +33 (0)2 35 12 14 14

PL TABELA PROGRAMÓW 5019 312 11174
PL TABELA PROGRAMÓW 5019 312 11174
20/03/2015 - docs.whirlpool.eu
PL TABELA PROGRAMÓW A. Programator B. Przyciski wyboru funkcji C. Przycisk Temperatura  D. Przycisk Opóznienie startu  E Przycisk Prêdko[ wirowania  F. Przycisk Start(Pauza)  G. Przycisk Anulowanie/Wypompowanie  H. Wy[wietlacz I. Przycisk Ulubione  Wskaznik fazy programu Wskaznik Drzwiczki otwarte  Program TemperaEtykiety tura Mieszane Biale Jasne B G Bawelniane Ciemne Pranie Û wstepne Pralka jest wyposa|ona w ró|ne automatyczne funkcje zabezpieczajce, któ

HD Flow - TomTom
HD Flow - TomTom
22/02/2017 - www.tomtom.com
HD Flow Des informations trafic en temps réel Des informations trafic précises et en temps réel sont de plus en plus précieuses, compte tenu du nombre grandissant de véhicules empruntant le réseau routier disponible et de l'augmentation des risques de saturation du trafic. Grâce à TomTom HD Flow, les routes sont contrôlées en permanence afin de déterminer la vitesse moyenne des véhicules sur chaque portion de route et répondre ainsi aux besoins d'un grand nombre d'utilisateurs : "