The mystery of “Debug adapter process has terminated unexpectedly”

Interesting story happened to me the other day. Visual Studio Code, my primary C# editor on Linux, suddenly stopped working. Well, it’s debugger did. Whenever I put a breakpoint and started debugging, nice friendly message would appear in the top of the editor, saying:

Debug adapter process has terminated unexpectedly

And it all worked just fine before. As recently I installed a whole bunch of new tools on my Ubuntu, such as lldb, perf and lttng, I played with removing and re-adding them again, reinstalling VS Code itself, but nothing seemed to help. Surprisingly, another C# IDE that supports Linux – JetBrains Rider – also failed to debug the projects I needed (worked fine with few others, though), so I had no other choice but try to get to the bottom of it, or at least make VS Code work again.

Looking for dump files

Whenever a process crashes on my Ubuntu, it will leave a dump file. That’s the way I configured it a while ago and now this behavior would become handy again. After all, “Debug adapter process has terminated unexpectedly” sounds like something that would leave a core dump or two. And you know what, it did! I found bunch of them right in VS Code’s C# extension folder:

Failing executable – vsdbg-ui – was right next to them, so it was interesting to take a look at what brought it down.

Checking the stacktrace

OK, let’s take lldb debugger and open that dump:

The first command I usually run inside of a dump is backtrace – a call stack of current thread. That’s usually a good place to start.

This is excellent! I mean process failed, that’s sad, but libcoreclr.so inside of call stack means it’s .NET Core process and FinalizerThread with DispatchManagedException indicate something happened in one of object destructors and a managed exception was thrown, which eventually terminated the whole process.

As we’re talking about .NET Core, we’d better enable SOS plugin and start dealing with its managed threads and objects.

Going managed

The simplest way to get SOS plugin is to pick one from .NET Core SDK folder, which I just happened to have. After enabling it, I could run something like sos PrintException and see what exactly was that exception about.

OK, it’s complaining that libsos.so missing in current folder. No problem, I know .NET Core SDK has one, so I just copy it here and retry:

“The current is unmanaged”. I can deal with that, but for now, how many managed threads we’re talking about?

Analyzing threads

And here’s another huge luck. Very first thread, (OSID – 2c42) had System.NotSupportedException in it. There’s also memory address next to exception, 00007fa5e42303a0, so we can look inside:

It’s _message property is not empty and we can dump it as well:

OK, so something doesn’t support “GetObjectID” method. I wonder what exactly. Let’s get back to the thread.

Getting exception call stack

The last time PrintException didn’t work, because current thread was unmanaged. But we can link managed thread to current unmanaged one and make SOS happy.

Theoretically, if I connect CLR thread #1 (OSID – 2c42) to current unmanaged thread #1, it might do the trick.

Great! The stack is long enough to start googling. .NET Core is open source, after all.

Working with sources and the great reveal

It was fairly easy to find InteropExtensions.cs which holds GetObjectID method, and indeed it was throwing the exception. However, that wouldn’t explain why did it start failing now and how to stop it.

Going up the call stack into __ComObject.Cleanup was more useful, because the only lines that call GetObjectID are lines like these:

It’s some sort of logging which, according to IsEnabled method name, may or may not be enabled. Going into InteropEventProvider.IsEnabled doesn’t lead to an insight, but triggers an idea.

There’s EventSource inside, and one of the things that also did change on my Ubuntu recently was setting an environmental variable: COMPlus_EnableEventLog. It enables tracing events in .NET runtime and I used it for troubleshooting (and a blog post) at some point. The switch doesn’t discriminate and should affect vsdbg-ui as well. What if that’s the problem?

A minute later I remove that environmental variable and VS Code debugger works again.

Conclusion

That’s one of the stories that don’t have satisfying happy ending and still leave many questions unanswered. Why it was using a library compiled for WinRT? There are actually two InteropExtensions implementations in CoreCLR repo and the other one has GetObjectID implemented. Why wasn’t it used instead? Why couldn’t I repeat the issue on brand new Ubuntu and “Hello World” app?

On the other hand, I’ve got my debugger back. You can’t even imagine how stupid it feels to debug with Console.WriteLine.

One thought on “The mystery of “Debug adapter process has terminated unexpectedly”

Leave a Reply

Your email address will not be published. Required fields are marked *