Managing the Machine Learning Lifecycle with MLflow and R

# Managing the Machine Learning Lifecycle with MLflow and R
## <a href='http://kevinykuo.com/talk/2018/10/sais-eu/'>kevinykuo.com/talk/2018/10/sais-eu</a>
### Kevin Kuo <br /> <span class="citation">@kevinykuo</span>
### October 2018

---

# This talk

.Large[Repurposed from a **sparklyr**/NLP talk at the last minute.]

.Large[In case you're here for that, we'll give a 30 second update in a little bit!]

---

# About me

.Large[
- Software Engineer, Overhyped Technologies @RStudio
  - (big data, deep learning, AI, etc.)
- Options trader -> math grad student -> actuary -> management consultant -> data scientist -> software engineer
]

.Large[
- I also care about things that matter, like buildings tools to help R users put models into production.
]

---

![](img/blogpost.png)

---

# Structured streaming with Shiny integration

```r
reactiveCount <- stream_read_text(sc, "source/") %>%
  ft_tokenizer("line", "tokens") %>%
  ft_stop_words_remover("tokens", "words") %>%
  transmute(words = explode(words)) %>%
  filter(nchar(words) > 0) %>%
  group_by(words) %>%
  summarize(n = n()) %>%
  arrange(desc(n)) %>%
  filter(n > 100) %>%
  reactiveSpark()

ui <- fluidPage(plotOutput("wordsPlot"))

server <- function(input, output) {
  output$wordsPlot <- renderPlot({
    reactiveCount() %>%
      head(n=10) %>%
      ggplot() + aes(x=words, y=n) + 
      geom_bar(stat="identity")
  })
}
```
]

---

# Monitoring and interrupting jobs

Now you can hit `<STOP>` in the IDE without breaking your session!

---

# k8s

```r
sc <- spark_connect(config = spark_config_kubernetes("k8s://hostname:8443"))
```

.Large[
Blog post: [blog.rstudio.com/2018/10/01/sparklyr-0-9/](https://blog.rstudio.com/2018/10/01/sparklyr-0-9/)

Documentation: [spark.rstudio.com](http://spark.rstudio.com/)
]

.Large[
Also check out **parsnip** (successor to **caret**) integration: [github.com/topepo/parsnip](https://github.com/topepo/parsnip) (note: experimental)
]

---

# Back to the our regularly scheduled programme...

---

# .Large[MOTIVATION]

---

# `theme: productionization`

---

[youtu.be/-K9SjrWpeys](https://youtu.be/-K9SjrWpeys) [@josh_wills](https://twitter.com/josh_wills)

---
# Gross generalization

.Large[Data scientists using R primarily tend to be stats/natural sciences/social sciences types who picked up programming...]

.Large[`%in% room`]

.Large[...so they're clueless when it comes to software engineering principles and hardening models 🤷.]

---

# Python users, on the other hand...

![](img/reticulated_python.png)
]
---

<img src="img/chopper1.jpg" width="100%" />
]

---

Spark ML, xgboost, random CRAN packages, Keras, TensorFlow, scikit-learn, H2O, ...

![](img/ml_everywhere.jpg)

---

# `theme: reproducibility`

---

![](img/reproducibility-graphic-online1.jpeg)

[nature.com](https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970)

---

# These problems had already been solved...

# for some.

---

[eng.uber.com](https://eng.uber.com/michelangelo/#)
]

[code.fb.com](https://code.fb.com/core-data/introducing-fblearner-flow-facebook-s-ai-backbone/)
]

---

# But what if you're *not* a big tech company? 🤔

---
class: center, middle

---

# Efforts in the R ecosystem (excerpt)

.Large[
- *mleap*: MLeap integration for sparklyr for serializing Spark ML pipelines
- *tfruns*: Track and Visualize Training Runs (for TF and Keras)
- RStudio Connect: Native TF model deployment, arbitrary R models via plumber
- RStudio Connect: Reproducible report publishing and sharing
- **mlflow: interface to MLflow**
]

.Large[We likely won't ever solve everyone's problems with one framework, but we should be able to standardise on 90% of the problems and have good/generally accepted guidance on the rest.]

---

# MLflow

.Large[
- **Tracking:** keep track of your parameters, notes, and metrics for experiments.
- **Project:** bundle your project and environment so others can reproduce your results.
- **Model:** serialize and package your scoring function for serving locally and on the cloud.
]

---

# Demo time!

---

# Recap

- MLflow is cool and you should check it out. Share feedback and use cases!
  - [github.com/mlflow/mlflow](https://github.com/mlflow/mlflow/tree/master/mlflow/R/mlflow). For R issues tag `@kevinykuo` and `@javierluraschi`.

- R package will be on CRAN soon.

- Link to this talk (incl. demo to be uploaded after talk): [kevinykuo.com/talk/2018/10/sais-eu/](https://kevinykuo.com/talk/2018/10/sais-eu/)