Problems with APIC on QEMU

General discussion about Visopsys.
User avatar
andymc
Posts: 589
Joined: Tue Nov 16, 2010 7:20 pm

Re: Problems with APIC on QEMU

Post by andymc »

Hi guys,

It does seem to be true that, for example, the IDE driver is missing APIC interrupts in the Qemu 2.0.0 and 2.1.0 releases.

However, from browsing the Qemu changelogs, it does look like they've been actively making fixes to their APIC implementation, and Qemu 2.2.1 (the latest) works with Visopsys.

Since VMware, VirtualBox, Qemu 1.0, Qemu 2.2.1, and real hardware work well with the new Visopsys APIC code, I think I'm going to leave this one alone, rather than delve deeply into getting my APIC driver to work with particular, non-current 2.x versions of Qemu.

Unless someone thinks they can change my mind, I'll just tell people to upgrade their Qemu :-)
User avatar
ronaldlees
Posts: 120
Joined: Tue May 20, 2014 5:19 pm

Re: Problems with APIC on QEMU

Post by ronaldlees »

Hi Andy!

I read your post about Qemu 2.2.1, so I fetched it for Debian Wheezy (that's the platform where I was having trouble running Visopsys). I'm happy to report that Visopsys works great on Debian, now that I've upgraded the Qemu to version 2.2.1. So - no objections from me about forgetting about the earlier versions. Sounds like the case is closed - it was a Qemu problem.
User avatar
andymc
Posts: 589
Joined: Tue Nov 16, 2010 7:20 pm

Re: Problems with APIC on QEMU

Post by andymc »

Great, thanks for trying it, Ron. It's good to get verification from someone else.
User avatar
ronaldlees
Posts: 120
Joined: Tue May 20, 2014 5:19 pm

Re: Problems with APIC on QEMU

Post by ronaldlees »

I've been looking at Visopsys's new touchscreen feature, and wanted to use QEMU so I'd have gdb debugging. I think we've documented that only QEMU 2.2.1 works with Visopsys.

Running QEMU 2.2.1 with Visopsys 0.8_pre, from the downloaded ISO works OK. But, if I build Visopsys with debug info enabled (-g), use the debug target, and modify the makefiles slightly to prevent stripping, then when I run the newly built Visopsys with QEMU 2.2.1, I see the interrupt issues (with same symptoms as all the other versions of QEMU).

So, perhaps the debug build introduces some latency that triggers this?
User avatar
ronaldlees
Posts: 120
Joined: Tue May 20, 2014 5:19 pm

Re: Problems with APIC on QEMU

Post by ronaldlees »

Finally, I'm making some sense of this QEMU thing. Well, I'm making sense of it until something else pops up, in any case. But - we've had some progress perhaps.

I noted in the last message that I was seeing the symptoms of our QEMU related interrupt issues with "debug" target builds. Andy's prebuilt ISO for version 0.80_pre1 works fine. But, building Visopsys here from source, with debugging info included, resulted in the "interrupt" symptoms. Then, this morning, even the regular release target, built here from sources, resulted in an ISO that exhibited the now well known symptoms in QEMU.

I mounted both ISOs (Andy's ISO was downloaded, and a release version ISO was built on my end). I did a file-by-file comparison. The source was identical (except for a "key" file related only to the partition manager project). Andy's ISO is bigger mainly because he includes the "docs" directory in his distributed ISOs. Then, I did a diff -a -r -U3 -T on the binaries on each ISO mount. The binary files were found to be different, for a few reasons including that different compilers may have been used to compile them. I looked through the diff, and noticed that Andy is using Ubuntu Linaro 4.6.3-ubuntu5 to build Visopsys. My own system was using GCC4.8.

So, I did:

Code: Select all

apt-get install g++-4.6
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 50
When using a compiler with the same version as Andy's (GCC4.6) I get good ISOs for both release and debug. No "interrupt" symptoms!

This may be what others will want to do to get QEMU going. Now, I'm wondering if some of the other QEMU versions will work with a GCC4.6-built Visopsys.
User avatar
ronaldlees
Posts: 120
Joined: Tue May 20, 2014 5:19 pm

Re: Problems with APIC on QEMU

Post by ronaldlees »

With the Visopsys build pretty solid, and QEMU working with it, my next big thing was to figure a way to switch the current thread in a GDB session. That's something that's real easy to do when debugging userland programs running natively and locally.

I realized that all of the GNU debugger's process and thread functionality was tied into specific attributes of the operating systems (mainly of the *nix/posix types) - such as libthread and fork, and Visopsys has neither of those. So, there are no thread events for the debugger. GDB options like schedule-multiple, non-stop, attach, and so forth have no meaning in this context.

Issuing an "info threads" GDB command while connected to QEMU, while debugging Visopsys, just returns an error. The GDB server stub in QEMU supports debugging whatever process it started with, but no thread info comes back when Visopsys is the virtualized OS. Within the initial process and "thread" - it works surprisingly well, but it's not as useful as userland program debugging. One thing that is an issue is that the kernel itself gets stopped and started again and again. Sometimes, this interferes with getting useful information. I think this setup would probably work better in a simple bare-metal or RTOS situation, with a single task. The problem with starting and stopping causing interference might be mitigated somewhat by using conditional breakpoints/watchpoints instead of hard breakpoints.

I end up debugging the kernel in one session, but then have to stop and reconnect in order to debug an application program in another session. I tried to attach to two processes at the same time (doable in a native/local environ) - but the GDB server stub in QEMU allows only one connection at a time. In retrospect, that's reasonable. I think it works in one-foot-after-the-other mode. The stub is not very fancy. In the unfancy mode, you have one activity per CPU, and there's only one CPU.

It does work decently to debug code in this fashion, depending on what you want to do. When debugging an app (today I was looking at hostname) - I can just step over any kernel function calls, make a note for any that fail, and then restart GDB with the debugger pointed at the kernel.

Have you ever thought about putting a serial debug output on Visopsys?
User avatar
andymc
Posts: 589
Joined: Tue Nov 16, 2010 7:20 pm

Re: Problems with APIC on QEMU

Post by andymc »

ronaldlees wrote:So, I did:

Code: Select all

apt-get install g++-4.6
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.6 50
When using a compiler with the same version as Andy's (GCC4.6) I get good ISOs for both release and debug. No "interrupt" symptoms!

This may be what others will want to do to get QEMU going. Now, I'm wondering if some of the other QEMU versions will work with a GCC4.6-built Visopsys.
Aha, excellent work, detective!

I try to avoid upgrading my workstation OS and compilers too often, because with GCC - unfortunately - issues do tend to pop up. Those guy do seem to tinker with things quite a lot, and I sometimes have to do some difficult/unpleasant debugging as a result.

I'd like to kick this can down the road, but if you guys are all using newer compilers, then I suppose I should too.
User avatar
andymc
Posts: 589
Joined: Tue Nov 16, 2010 7:20 pm

Re: Problems with APIC on QEMU

Post by andymc »

ronaldlees wrote:I end up debugging the kernel in one session, but then have to stop and reconnect in order to debug an application program in another session. I tried to attach to two processes at the same time (doable in a native/local environ) - but the GDB server stub in QEMU allows only one connection at a time. In retrospect, that's reasonable. I think it works in one-foot-after-the-other mode. The stub is not very fancy. In the unfancy mode, you have one activity per CPU, and there's only one CPU.

It does work decently to debug code in this fashion, depending on what you want to do. When debugging an app (today I was looking at hostname) - I can just step over any kernel function calls, make a note for any that fail, and then restart GDB with the debugger pointed at the kernel.
Ronald my friend, all this stuff you do is pretty amazing - a lot of it is beyond my sphere of knowledge. I'm a toddler when it comes to using debuggers.

Yes, Visopsys' threads are proprietary. They're quite lightweight; they have their own process structure and stack, and that's about it.

The main kernel process itself doesn't do anything after initialization, it just blocks on the startup program (typically 'imgboot' or 'login'). It does have several threads that manage various subsystems, distributing input and GUI events, and whatnot. When userspace programs call the kernel API, execution happens in the caller's context.
ronaldlees wrote:Have you ever thought about putting a serial debug output on Visopsys?
I might have thought about it if I was like Ron the debugging ninja, but no, not really :mrgreen: Would it be exceptionally useful?
User avatar
ronaldlees
Posts: 120
Joined: Tue May 20, 2014 5:19 pm

Re: Problems with APIC on QEMU

Post by ronaldlees »

I might have thought about it if I was like Ron the debugging ninja, but no, not really :mrgreen: Would it be exceptionally useful?
LOL - If there were any ninjas on this end, I'd have submitted a bug patch by now. But, seriously, I'm slowly getting the parameters lined up so that I might be able to make myself useful. I have to do it in my sphere of knowledge, which is userland of course. Well, I guess I wouldn't absolutely have to do it that way. Maybe I'm getting lazy.

About the serial capability, it's more about convenience than actual real benefit. For instance, in QEMU you can do:

Code: Select all

qemu-system-i386 -hda blahblah -cdrom blahblah -boot d -s -S -sdl -serial file:seriallog.txt
That -serial format will take all of the emulated machine's serial output (#1) and put it in the log file on the host. But - yeah - it's not a must have sort of thing.

- Ron
Post Reply