How to set timing
 

Is there a "scientific" way to set up timing for a modified setup?

Everywhere I've read seems say it's done by trial and error. How do I know How far to go without detonating?

the scientific way is called MBT. with the rotary we want to be even more conservative than the piston people, because knock is harder to detect, and seems to do damage more quickly, but concept is the same

EFI University Electronic Fuel Injection Tuning :: View topic - tuning for MBT on a dyno dynamics dyno

actually to expand on this a little, in the NA world this is easy, you go to dyno, and add timing until it stops gaining power, and then back off a couple degrees.

in the turbo world, it will take time and discipline. knock can be hard to detect too. i would suggest starting with a partial load, and lower rpm, to find knock, and then build from there.

i would start at a really low rpm, like 1500-2000, as knock can be found safely, although it is a problem that when the RPM goes up, knock is harder to find and more dangerous