How to Fix 'Could not find bin\roslyn\csc.exe'

One working project failed to compile after I made a fresh checkout. The same commit that before worked without a problem now got me this compiler error:

Could not find a part of the path C:\*****\bin\roslyn\csc.exe

This error only happened when I compiled the ASP.Net application of my project, the command line applications worked as expected. It took me a while to pinpoint the problem and finding a solution was another challenge.

I finally got the problem soled as I installed the Microsoft.Net.Compilers package in my web project:

Install-Package Microsoft.Net.Compilers -Version 3.3.1

It had to be exactly version 3.3.1. Version 3.4 did not work and created a whole bunch of other error messages. If you have the same problem, try to use this version as well if newer versions do not work.