The thingy mostly works now.
The FF2 descriptor made the real difference in finding out what the WIndows PID driver really accepts. I've done full scale testing with FEdit, FConst and some preliminary testing with IL2 game too. With FEdit, I can play effects mostly fine. In IL2, there are some weird glitches and issues in the effects making it not really playable yet, but I consider this level of achievement as a proper proof of concept
Also, no need for that registry magic anymore - plug the joystick in and Windows will automatically generate correct OEM/FFB registry entries it likes to have. The decisive factor was the FF2 descriptors and that I needed to report a correct PID state reports when requested. None of which seem to be documented anywhere in sufficient detail - as you might guess. The FF2 descriptor also has several features not really supported by FFP (like phase or offset of periodic forces).
Creating effects, playing and stopping them work. The Windows USB-protocol implementation did (surprise surprise) not actually match the one described in USB.org's documents, but had some shortcuts that caused some pulling of hair (or what's left of it) at a time.
Almost all effect types now work when played using e.g. FEdit; constant effect, sine, triangle, square, spring, friction, inertia almost as much as FFP can support them. There were lots of nasty number format conversions and FFP limitations to stumble into. Ramps are not fully working yet - FFP also has very strict limits on what kind of ramps it actually can perform (only rampup from MIN to MAX or rampdown from MAX to MIN - nothing in between).
FFP can play up to 10 effects simultaneously even tho it can hold 40 definitions in its memory. Both properties are now taken into account. The adapter now needs to keep all 40 definitions in memory (taking a lot of RAM) since USB data updates are not always sent to MIDI immediately and need to be cached in the adapter. Some optimization could be done but ATmega32U4 can just take it all already now.
Most on-fly updates to already playing effects are also now working. E.g. changing direction or magnitude of a force already playing work. There are some limitations in FFP as it does allow changing e.g. magnitude of already playing constant force, but not periodic forces.
All that after tens of hours of tedious trial and error, half-educated guesses and even a few reboots due blue-screens in XP. Windows 7 64-bit works too.
FFB also seem to affect the joystick position reports by making some of the values jump erratically every now and then. I remember seeing some button line activity when playing effects - possibly it reports to PC whether an effect is playing or not or something like that. At the moment, I just ignore all that data, but it might be causing the erratic jumps (and random reported button clicks) in the input reports.
I suspect that there are also issues with the way I use LUFA-library too since I run into random issues if I get a large burst of output data going to the joystick in a short period of time. There seem to be also some timing issues with the MIDI-data to joystick. There are also some timing issues in replying to some USB-requests too quickly from the adapter to PC. I hope to get these solved by getting in touch with some LUFA-experts. It now looks very plausible to have a generic sample force feedback joystick project added to LUFA too.
I have already updated some of the MIDI-protocol documentation parts in the project Wiki and I'll post the new version of the code in the coming days too.
Thank you to all involved - especially Grendel and Ulao - for the help and support to get this thing already this far! I hope we can finalize it to be actually playable with any decent game.
BTW: I just got a second FFP with €10 to make sure the efforts are not wasted for me if one happens to break...