Creating an ROC curve and identify the optimal threshold value for a detection method
Dataset for ROC Curves XLS56.5 KB
-
Life Survival analysis software
- View Life page
- Download Trial version
- See users' feedback
-
System configuration
- Windows:
- Versions: 9x/Me/NT/2000/XP/Vista/Win 7
- Excel: 97 and later
- Processor: 32 or 64 bits
- Hard disk: 150 Mb
- Mac OS X:
- OS: OS X
- Excel: X, 2004 and 2011
- Hard disk: 150Mb.
- Windows:
Benefits
-
Easy and user-friendly
Easy and user-friendly XLSTAT is flawlessly integrated with Microsoft Excel which is the most popular spreadsheet worldwide. This integration makes it one of the simplest available tools to work with as it utilizes the same philosophy as Microsoft Excel. The program is accessible in a dedicated XLSTAT tab. The analyses are grouped into functional menus. The dialog boxes are user-friendly and setting up an analysis is straightforward.
-
Data and results shared seamlessly
Data and results shared seamlessly One of the greatest advantages of XLSTAT is the way you can share data and results seamlessly. As the results are stored in Microsoft Excel, anyone can access them. There is no need for the receiver to have an XLSTAT license or any additional viewer which makes your team-work easier and more affordable. In addition, results are easily integrable into other Microsoft Office software such as PowerPoint, so that you can create striking presentation in minutes.
-
Modular
Modular XLSTAT is a modular product. XLSTAT-Pro is a core statistical module of XLSTAT which includes all the mainstream functionalities in statistics and multivariate analysis. More advanced features contained in add-on modules can be added for specific applications. This way you can adapt the software to your needs making the software more cost-efficient.
-
Didactic
Didactic The results of XLSTAT are organized by analysis and are easy to navigate. Moreover useful information is provided along with the results to assist you in your interpretation.
-
Affordable
Affordable XLSTAT is a complete and modular analytical solution that can suit any analytical business needs. It is very reasonably priced so that the return of your investment is almost immediate. Any XLSTAT license comes with top level support and assistance.
-
Accessible - Available in many languages
Accessible - Available in many languages We have ensured XLSTAT is accessible to everyone by making the program available in many languages, including Chinese, English, French, German, Italian, Japanese, Polish, Portuguese and Spanish.
-
Automatable and customizable
Automatable and customizable Most of the statistical functions available in XLSTAT can be called directly from the Visual Basic window of Microsoft Excel. They can be modified and integrated to more code to fit to the specificity of your domain. Adding tables and plots as well as modifying existing outputs becomes easy. Furthermore, XLSTAT includes some special tools on the dialog boxes to generate automatically the VBA code in order to reproduce your analysis using the VBA editor or to simply load pre-set settings. This effortless automation of routine analysis will be a huge time saver on your part.
ROC curve
This method was first developed during World War II to develop effective means of detecting Japanese aircrafts. It was then applied more generally to signal detection and medicine where it is now widely used.
The problem is as follows: we study a phenomenon, often binary (for example, the presence or absence of a disease) and we want to develop a test to detect effectively the occurrence of a precise event (for example, the presence of the disease).
If the test is quantitative (possibly ordinal), for example, a concentration of a molecule, we will try to determine from what concentration can a patient be considered as ill. The ROC curves and indices calculated here will help us make the best decision.
Dataset to generate a ROC curve
An Excel sheet with both the data and results used in this tutorial can be downloaded by clicking here.
The data correspond to a medical experiment during which 50 patients, among which 20 are sick, are submitted to a screening test where the concentration of a viral molecule is being measured.
Setting up of a ROC curve
Once XLSTAT has been started, select the XLSTAT-Life / ROC Curves command, or click on the corresponding button of the XLSTAT-Life toolbar (see below).
When you click on the button, a dialog box appears. Select the data that correspond to the event data and enter the code that is associated to positive cases.
Then select the data that correspond to the diagnostic test that we are evaluating and specify what type of rule should be used to identify the threshold value below or above which the test should be considered positive.
We choose here to consider that the test is positive if the concentration is greater than or equal to a value to be determined.
In the Options tab, you can specify the method for calculating the confidence intervals.
XLSTAT is the software offering the widest choice. The defaults are those most recommended.
In this tab, you can also assign a cost to the various cases. We wish to strongly penalize diagnostic errors and more particularly the case where the sick patients are not detected.
In the Charts tab, we choose to display a decision plot based on costs.
When you click OK, the computations are done and the results are displayed.
interpretation of a ROC curve
The first table displays the descriptive statistics for the test variable, here the concentration, followed by the statistics for the event variable, here the disease. The prevalence observed on the sample is equal to 0.4.
The ROC is then displayed. To each small square corresponds an observation.
The "ROC Analysis" that follows, displays for each possible threshold value, the value of the various performance indices. For example, if we decide to declare a patient sick when the concentration is greater than or equal to 0.98, the sensitivity is 0.95, the specificity of 0,733 and the cost is 61. For more details on the various indices, you can check the tutorial on sensitivity and specificity analysis.
A graph constructed using this table is then displayed. He can see the evolution of counting TP (true positives), TN (true negative), FP (false positives) and FN (false negatives) depending on the chosen threshold value.
The decision plot allows to choose the threshold value that minimizes the cost. To see what is the minimum threshold value on the chart, just leave your mouse over the corresponding point. This value corresponds to a concentration of 0.98, as we had identified earlier in the table analysis ROC.
The last series of results allows the study the area under the curve (AUC). The AUC and its confidence interval are calculated. The test that compares the AUC to 0.5 allows to check if the diagnostic test is more powerful than just a random rule. In our case, the test is very powerful and the AUC is significantly different from 0.5.
The comparison of the AUC is also a way to compare different diagnostic tests. XLSTAT can compare as many tests as you like.