How to use Microsofts ML.NET in HedgeTools
 June 5, 2019
Charles R. Brauer

The Problem

In Machine Learning the most popular computer language is Python. This is a great language for researching classification and regression models. However, Python is way too slow to be used in production.

The Solution

The most popular language for Microsoft developers is C#. But Python and C# do not play well together.

The solution for Machine Learning in a Microsoft environment is Microsoft’s ML.NET. Now we can produce models in C# and execute the trained model to produced predictions with an order of magnitude in speed over Python. This article shows a test of how this can be done.

Launch Visual Studio 2019 and create a new “Test” project. Then select “Console App (.NET Core)” from the menu screen. After the project is created, we then add a “Custom scenario” machine learning model to the application. All that is required is to right click on the project name (Test) and choose Add and then “Machine Learning”. Your screen will look something like:

Figure 1

Next a screen will appear that allows you to select the dataset:

Figure 2

When you click on the “Train” link you will see the following screen:

Figure 3

Click the “Start Training” button and the fun begins. When you see the training traces, you may be wondering where the training is done. It’s on an Azure server. You can open a developers account that is free. One of the limitations is CPU time. It’s one hour. And in this test that was enough to give a very good model.

Figure 4

Step 4 is the Evaluation of the training results. Notice in the Output window you will see the where the model files have been stored.

Now click on Step 5 Code. Your screen should look like:

Figure 5

Notice the new Project “TestML.ConsoleApp”.  The MLModel.zip file contains a “.dll” file that is your trained model.  Now we can use this model to compute some metrics.

The next step is to delete Test project. Your Solution Window should now look like:

Figure 6

The complete program listing for Program.cs is:

using System;
using System.Collections.Generic;
using System.IO;

using Microsoft.ML;
using TestML.Model.DataModels;

namespace TestML.ConsoleApp {

public static void Predictor(string predictors,
out int predicted,
out double probability) {

var mlContext = new MLContext();
const string modelPath = @"../../../../TestML.Model/MLModel.zip";
var mlModel = mlContext.Model.Load(modelPath, out _);
var predEngine = mlContext.Model.CreatePredictionEngine(mlModel);
var modelInput = new ModelInput();
var fields = predictors.Split(',');
modelInput.BoxRatio = Convert.ToSingle(fields[0]);
modelInput.Thrust = Convert.ToSingle(fields[1]);
modelInput.Velocity = Convert.ToSingle(fields[2]);
modelInput.OnBalRun = Convert.ToSingle(fields[3]);
modelInput.VwapGain = Convert.ToSingle(fields[4]);
var prediction = predEngine.Predict(modelInput);
predicted = prediction.Prediction ? 1 : 0;
probability = 0.0;

}

private static void RunTest() {

int predicted, false_positive, true_negative, false_negative;
var true_positive = false_positive = true_negative = false_negative = 0;
double goal_met = 0.0, goal_failed = 0.0, probability;
var data = new List();
var url = "https://raw.githubusercontent.com/CBrauer/CypressPoint.github.io/master/rockettest.csv";
var req = (HttpWebRequest)WebRequest.Create(url);
var resp = (HttpWebResponse)req.GetResponse();
var reader = new StreamReader(resp.GetResponseStream());
reader.ReadLine(); // Ignore the header

while (!reader.EndOfStream) {   var line = reader.ReadLine();
  data.Add(line);
}

var loop = 0;

foreach (var predictors in data) {
  Predictor(predictors, out predicted, out probability);
}
var fields = predictors.Split(',');
var actual = Convert.ToInt32(fields[5]);

if (actual == 1) goal_met++; else goal_failed++;
if (predicted == 1 && actual == 1) {
  true_positive += 1;
}
if (predicted == 1 && actual == 0) {
  // False Positive. The model predicted a gain, but we got a loss.
  false_positive += 1;
}
if (predicted == 0 && actual == 0) {
  // True Negative. The model predicted a loss, and we got a loss.
  true_negative += 1;
}
if (predicted == 0 && actual == 1) {
  // False Negative. The model predicted a loss, but and we got a gain.
  false_negative += 1;
}
Console.Write(++loop + ": " + true_positive + ", " + false_positive +
              ", " + false_negative + ", " + true_negative +
              ", " + false_negative + ", " + true_negative +
              "\r") }
double recall, f1score;
var precision = recall = f1score = 0.0;
var denom = Convert.ToDouble(true_positive + false_positive);
if (denom > 0.0) precision = true_positive/denom;
denom = Convert.ToDouble(true_positive + false_negative);
if (denom > 0.0) recall = true_positive/denom;
if (precision + recall > 0.0) {
  f1score = 2.0*(precision*recall)/(precision + recall);
}
precision = Math.Round(precision, 4);
recall = Math.Round(recall, 4);
f1score = Math.Round(f1score, 4);
Console.WriteLine(" No. of True Positive......" + true_positive + "/" + goal_met);
Console.WriteLine(" No. of False Positive....." + false_positive);
Console.WriteLine(" No. of False Negative....." + false_negative);
Console.WriteLine(" No. of True Negative......" + true_negative + "/" + goal_failed);
Console.WriteLine(" Precision................." + precision);
Console.WriteLine(" Recall...................." + recall);
Console.WriteLine(" F1 Score.................." + f1score);
Console.WriteLine("End of process, hit any key to finish");
Console.ReadKey();

}

private static void Main() {
  RunTest();
}

}

}

}

The Result

When the program is run, we get the following results:

Figure 7

Comments

Preliminary timing tests show that the function:

  predEngine.Predict(modelInput);

is more than 10 times faster than the equilevent Python code. That’s great!

Also, I have been workiing on Python models for the last two years. The best models from:

• TensorFlow
• TPOT
• SKLearn
• H2O

are significantly worse than this Classifier from Microsoft.
I have never seen results this good. This is truly a breakthrough.

I challange you to produce better results.

CBrauer@CypressPoint.com