Previous Thread
Next Thread
Print Thread
Rate This Thread
Hop To
Page 1 of 3 1 2 3
#4477680 - 06/12/19 12:12 AM Need Help With Stutter Testing  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Hi All,

I continue to work on making my WOFF performance the best it can be. Now I need some help from this fine community to help me test and hopefully narrow down the cause of those intermittent and occasional stutters that cause the action to momentarily pause. I'm not talking about initial stutters that can be cleared with some 360 panning of the pilot view prior to takeoff. Nor am I talking about the "spawn stutters" that occur often later in missions, presumably due to more planes entering the action (via GAI or simply later missions starts).

I'm excited to announce that I have finally managed to create a completely reliable and repeatable test scenario, which at least on my system, exhibits a very obvious stutter exactly 144 seconds into the mission (see the linked video). Now this is a slightly spooky number since 144Hz happens to be my refresh rate. However, I think that's pure chance. I think this stutter and other occasional stutters of this nature that I experience (aside from the types I mentioned earlier) are related to certain textures and or 3D models or other objects loading into memory. What memory? I'm not sure. Could be main system memory or graphics card memory. They could be due to some mod(s). They could be happening with a certain combination of game settings or detail settings. Since the test seems to be repeatable, I'm hoping to get some volunteers initially to run this scenario on their systems and let me know if they seem the same stutter instance @ 144 seconds. From there I'd love to get system and settings info so that I can start looking for potential differences and areas to focus on. I figure if I can resolve this then I can share and everyone benefits. It may prove simply too elusive, but I'd like to give it a go...


.... so, if you're up for helping me, please PM me and I'll send more details along with a link to the scenario. The test uses auto pilot and is very quick to set up. Big thanks in advance!

H



Best viewed full screen. Check the counter in the Z key display. The stutter occurs 144 seconds in and is quite obvious. Any other perceived lack of smoothness is due to video encoding. You'll have to take my word for that smile


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477681 - 06/12/19 12:20 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2015
Posts: 692
Stache Offline
Member
Stache  Offline
Member

Joined: Sep 2015
Posts: 692
Michigan, USA
Hi Harry,
PM Set.

BTW the video is "unavailable" for me.

Regards.


Insanity: doing the same thing over and over again and expecting different results. A. Einstein

(System Specs:)

I7-6700k OC 4.4GHZ, 16GB DDR4 3200Mhz; Gigabyte Gaming 7 MB, G1 OC'ed GTX980ti; Three-Acer XB271HU WQHD Gsync 144Mhz; Samsung 950-512GB NVMe SSD; WD 2TB-7200rpm; Cooler Master HAF XB EVO, Nepton 240M cooler, V1000 PS; Windows 10 PRO; VKB GunfighterPro Stick; Thrustmaster TPR Pedals; Saitek Throttle; Dual TM MFD panels; TrackIR 5
#4477684 - 06/12/19 12:40 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Video is now set to public. Thank you Stache! Instructions sent.


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477685 - 06/12/19 12:56 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
Normally anytime you get a "hitch" like that is when the game is "loading" something, be it new planes or new terrain.

I normally get this "hitch" at about 1,000 ft during take-off and then it is fine till it loads some more terrain or squadrons further into the game and I have a pretty decent PC.

You will find, the higher the Scenery Detail or Regional Air Activity you have in the Workshop, the more small "hitches" you will have during the game.

But then this is a 2002 "executable" and AnKor has done wonders just getting us this far with this old CFS3.exe.


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4477688 - 06/12/19 01:36 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Yep, agreed PR. So, from your perspective, do you think I'm on a wild goose chase, i.e this is simply not something that can be eliminated?


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477690 - 06/12/19 01:47 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2015
Posts: 692
Stache Offline
Member
Stache  Offline
Member

Joined: Sep 2015
Posts: 692
Michigan, USA
Perhaps not eliminated, but at least with a test that can replicate, we can see what, if anything affects it.

On my system it does replicate but at the 200-205 second mark.
Also on my system the flight turns to the right not the left as shown in Harry's video.

Last edited by Stache; 06/12/19 01:49 AM.

Insanity: doing the same thing over and over again and expecting different results. A. Einstein

(System Specs:)

I7-6700k OC 4.4GHZ, 16GB DDR4 3200Mhz; Gigabyte Gaming 7 MB, G1 OC'ed GTX980ti; Three-Acer XB271HU WQHD Gsync 144Mhz; Samsung 950-512GB NVMe SSD; WD 2TB-7200rpm; Cooler Master HAF XB EVO, Nepton 240M cooler, V1000 PS; Windows 10 PRO; VKB GunfighterPro Stick; Thrustmaster TPR Pedals; Saitek Throttle; Dual TM MFD panels; TrackIR 5
#4477692 - 06/12/19 02:44 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
OK thanks Stache. Perhaps if both you and PR can send me details of your AI settings for enemy aircraft I might see some similarities? UPDATE: NVM... you already did, thanks!

However, if we can't get similar results then it feels like a wild goose chase smile

Last edited by HarryH; 06/12/19 02:46 AM.

..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477694 - 06/12/19 02:55 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
I just set my settings the same as yours Stache, and they still turn left and the stutter is in exactly the same place for me. I wonder what the variable is that causes your AI to turn right instead?

Last edited by HarryH; 06/12/19 02:57 AM.

..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477696 - 06/12/19 03:13 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
OK without a truly repeatable test, this exercise is futile IMO. I'm going to go back to the drawing board and see if I can improve the scenario's ability to be consistent on different systems. Thank you Stache and PR for your help!

No more volunteers needed at this time

H oops


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477699 - 06/12/19 03:31 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2016
Posts: 596
BuckeyeBob Offline
Member
BuckeyeBob  Offline
Member

Joined: May 2016
Posts: 596
Ohio, USA
The relatively "long" stutter that occurs in your video at the 3:24 mark is caused by new scenery being loaded by the sim in response to the game engine and the compositebudgetscenery.xml file. In your video, most of the horizon is not quite far enough away to see the new scenery being loaded, but if you look very closely at the horizon on the very far right hand side of your video at the 3:23 mark, you just might be able to see some new scenery being loaded, just before the stutter occurs.

I have seen this effect quite clearly during testing of the blue triangles mod. Make sure you get up to at least 4000 feet with no clouds and a clear horizon, preferably a flat area over a big city or a heavily forested area, and then focus on the area between where you can see the last bit of detailed scenery and the horizon. Eventually, you should be able to see new scenery detail being added, followed just after by a fairly long stutter. I would attach a video showing you this, but I am an old coot and I haven't mastered Shadowplay/Youtube, so you will have to take my word for it. You may also notice slightly shorter stutters as new scenery is added to the "rings" nearer to your aircraft, but the worst stutter seems to occur in the most distant ring (probably because it has the largest circumference and takes longer to draw). Unfortunately, at this time I don't have a clear solution to this problem. Even reducing the object densities in the compositebudgetscenery.xml file doesn't eliminate this type of stutter. (There are other sources of stutters, too, unfortunately, but that is another matter). frown

#4477701 - 06/12/19 04:22 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Thanks for this explanation, BB. It certainly rings true to me, but I agree with your emoticon - that's too bad. In return, I'll send you a PM with step by steps to set up an easy way for you to capture video of the last 5 minutes of whatever you were doing. It's a pretty cool feature. I'd also be happy to run you through the process of uploading to YT smile


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477796 - 06/12/19 08:44 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2010
Posts: 1,196
33lima Offline
Member
33lima  Offline
Member

Joined: Sep 2010
Posts: 1,196
Belfast, NI
HarryH, do you have WoFF on a conventional hard drive or on a Solid State Drive? If your stutter is really related to stuff being loaded, an SSD should help, and they are neither expensive to buy nor difficult to install.

Following an ATAG forum suggestion, I recently eliminated horrible pauses in Cliffs of Dover every time a radio message was being played, by using a Steam function simply to move CloD from the HDD I had it on, to an SSD. Likewise, an SSD has long been known to reduce the regular hiccups you would get playing routes with denser than stock scenery in MS Train Simulator, which loaded the terrain and scenery in quite small chunks as you moved along the line; not seen since I moved to playing MSTS content in Open Rails AND moving it onto an SSD.

#4477815 - 06/12/19 10:09 PM Re: Need Help With Stutter Testing [Re: 33lima]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Originally Posted by 33lima
HarryH, do you have WoFF on a conventional hard drive or on a Solid State Drive? If your stutter is really related to stuff being loaded, an SSD should help, and they are neither expensive to buy nor difficult to install.

Following an ATAG forum suggestion, I recently eliminated horrible pauses in Cliffs of Dover every time a radio message was being played, by using a Steam function simply to move CloD from the HDD I had it on, to an SSD. Likewise, an SSD has long been known to reduce the regular hiccups you would get playing routes with denser than stock scenery in MS Train Simulator, which loaded the terrain and scenery in quite small chunks as you moved along the line; not seen since I moved to playing MSTS content in Open Rails AND moving it onto an SSD.


Of course, everything you've said about SSDs is correct. And they have been known to work wonders when eliminating this type of stutter from most any other game I can think of.

However - don't get me started about this game and (this specific) stutter - let me tell you this: Not only have I tried (what I call) "stupid fast" storage; that is to say a Samsung 950 Pro NVMe drive with a read speed in excess of 2200 Mbit/s...but I've also gone as far as creating a RAMdrive to load the entire sim to...

...and it makes absolutely. No. Difference. (And we're talking RAM that is easily many, many times faster than even a 'stupid fast' SSD.) Years ago, I tried 4 SSDs in a RAID 0 array (hardware based, not a crappy motherboard software implementation.) Nothing changed.

This game is plagued with (this specific) stutter. There is no solution, IMHO. It has nothing to do with settings, or how much someone spends on hardware, what FPS they claim to have, overclocking (really?), or anything else. CFS3 was known for this issue long ago, and it seems to be going nowhere, in spite of decades of technological improvement in hardware (like SSDs). People who have insisted they have no stutter, but then post videos which show this exact issue...or claiming they never drop below x FPS, but turn around and say they have the same recognizable hitch or glitch or whatever...

I build lots of gaming machines for 'competitive gamers'. Already done several this year. Every one comes with a 100% absolute money-back guarantee, yet no one's ever brought one back. They're out there, every single one of them, playing all sorts of different games, usually in some sort of competitive environment by some very picky players. Many of them overclocked to within an inch of their bloody little silicon lives...

...and no one has any complaints about stutter, or any performance issues at all. I must have some idea what I'm doing. At least they seem to think so...every single one this year so far was a referral from other people I've built for.

Unfortunately, as accurate and well-founded as your mention of faster storage may be...it doesn't solve this issue. TBH, I'm not even sure it improves the issue at all; where, as you pointed out, other games are often improved substantially. Even with the RAMdrive, the stutter I've tested repeatedly is unchanged.

People often don't see the stutter simply because they're not encountering the condition(s) which would trigger the stutter. This is why a repeatable test is important. I tried to do the same thing a long time ago and was blasted for it, by people who just couldn't admit this problem exists. Yet, somehow, oddly enough, it is the one subject that still comes up after all this time, and the one thing that (most) people do acknowledge, and the one thing there's been no solution for (well, maybe the cloud 'popping' issues, as well).

If there are no stutters, then why's it still coming up?

My observations, for whatever it's worth.

HH - best of luck, in all sincerity. But, as you have so accurately observed: A test that can be reliably duplicated is paramount, first and foremost.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4477856 - 06/13/19 03:38 AM Re: Need Help With Stutter Testing [Re: 33lima]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Originally Posted by 33lima
HarryH, do you have WoFF on a conventional hard drive or on a Solid State Drive? If your stutter is really related to stuff being loaded, an SSD should help, and they are neither expensive to buy nor difficult to install.

Following an ATAG forum suggestion, I recently eliminated horrible pauses in Cliffs of Dover every time a radio message was being played, by using a Steam function simply to move CloD from the HDD I had it on, to an SSD. Likewise, an SSD has long been known to reduce the regular hiccups you would get playing routes with denser than stock scenery in MS Train Simulator, which loaded the terrain and scenery in quite small chunks as you moved along the line; not seen since I moved to playing MSTS content in Open Rails AND moving it onto an SSD.


Thanks for the suggestion 33lima. Yes I have an SSD. Don't think that's the issue, though it's wonderful for speeding up the launch of the game itself smile


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477857 - 06/13/19 03:52 AM Re: Need Help With Stutter Testing [Re: kksnowbear]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Originally Posted by kksnowbear

If there are no stutters, then why's it still coming up?

My observations, for whatever it's worth.

HH - best of luck, in all sincerity. But, as you have so accurately observed: A test that can be reliably duplicated is paramount, first and foremost.


Thanks Kks, yes, I really thought I finally had a reliably repeatable test... and actually I think I kind of do. It's just that the obvious stutter occurs at different points in time for people due to a whole slew of factors - graphics settings, machine specs, etc. I am now convinced that this particular type of stutter is scenery related (as I think you also had surmised) and I am leaning toward the theory that PR and BB have proposed - it's down to the ancient CFS3 engine and it ain't going to be fixable. Ever.

I don't think anyone doubts your expertise, Kks. I certainly don't, and I'm sorry you got fried for raising the issue originally. These stutters are a real thing, no question. As players of this wonderful OBD enhancement to the original CFS3 we can either choose to live with it or walk away. For me. WOFF is still overwhelmingly an amazing experience, even with the occasional stutter. Much as I have hoped to find a fix for it though, it seems OBD have done the best they possibly can with the constraints of the old MS game engine. I will now choose to just live with it as-is. It's already hugely better than I've ever had it running before. Onward and upward smile

Thanks all.

Devs: feel free to close this thread if you want to smile


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477866 - 06/13/19 04:57 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2016
Posts: 596
BuckeyeBob Offline
Member
BuckeyeBob  Offline
Member

Joined: May 2016
Posts: 596
Ohio, USA
Originally Posted by HarryH
Thanks for this explanation, BB. It certainly rings true to me, but I agree with your emoticon - that's too bad. In return, I'll send you a PM with step by steps to set up an easy way for you to capture video of the last 5 minutes of whatever you were doing. It's a pretty cool feature. I'd also be happy to run you through the process of uploading to YT smile


Please go ahead and send me the information. I have a little test flight over the city of London that clearly illustrates the scenery stutter, if you would like to see it. In the meantime, I will poke around in the compositescenerybudget.xml file to see if I can reduce the stutter effect at all, but I am not very optimistic.

#4477868 - 06/13/19 05:07 AM Re: Need Help With Stutter Testing [Re: BuckeyeBob]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Originally Posted by BuckeyeBob
Originally Posted by HarryH
Thanks for this explanation, BB. It certainly rings true to me, but I agree with your emoticon - that's too bad. In return, I'll send you a PM with step by steps to set up an easy way for you to capture video of the last 5 minutes of whatever you were doing. It's a pretty cool feature. I'd also be happy to run you through the process of uploading to YT smile


Please go ahead and send me the information. I have a little test flight over the city of London that clearly illustrates the scenery stutter, if you would like to see it. In the meantime, I will poke around in the compositescenerybudget.xml file to see if I can reduce the stutter effect at all, but I am not very optimistic.


Happy to do so BB. Will get to it this weekend. Stand by...


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4477893 - 06/13/19 10:19 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
Since this is a WOFF topic and not a WOTR topic, I wonder why the Forum Moderators moved it to WOTR instead of leaving it in WOFF ?????????????


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4477904 - 06/13/19 11:51 AM Re: Need Help With Stutter Testing [Re: Panama Red]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Originally Posted by Panama Red
Since this is a WOFF topic and not a WOTR topic, I wonder why the Forum Moderators moved it to WOTR instead of leaving it in WOFF ?????????????


Interesting. Can't speak to why it was moved, but it bears mentioning that, as far as I can tell, this issue is no different in WOTR than in WOFF. There are videos posted that show it in WOTR just like in WOFF. And, actually, it may even be worse in WOTR, given the increased graphics load (this is not given, just a guess).

Also, to be clear: There's no doubt AnKor's efforts have really been helpful to these sims, and it is most appreciated I'm sure. However, I don't think that effort has changed this particular issue at all, and I have serious reservations about whether more DirectX work/DX11 is going to help with this specific issue. This is based on conversation with him directly.

As I've said many times, though: This is an area where I would genuinely love to be wrong.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4477941 - 06/13/19 04:41 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Mar 2003
Posts: 6,403
Polovski Offline
Polovski  Offline

Hotshot

Joined: Mar 2003
Posts: 6,403
Sorry, I moved it to the wrong forum - human error. Not sure it needed quite so many question marks, stuff happens wink

WOFF/WOTR switching between them does not help and there's the proof. Now moved to the WOFF Technical.

Last edited by Polovski; 06/13/19 04:41 PM.

Regards,

Polovski,
OBD Software, developers of immersive flight sims;
Wings Over Flanders Fields and Wings Over The Reich
http://www.overflandersfields.com
http://www.wingsoverthereich.com
#4477982 - 06/13/19 07:44 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
Thank you.

I figured it was an error, but I make mistakes too, so I was just checking. winkngrin


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478408 - 06/16/19 09:12 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2015
Posts: 692
Stache Offline
Member
Stache  Offline
Member

Joined: Sep 2015
Posts: 692
Michigan, USA
Hi,

Just wanted to finish documenting my results.

Here is a link to my video showing the flight turning to the right as opposed to the left in Harry's video
The issue can just barely be noticed around the 3:19 mark in the video.
This would be around the 2:02 mark in the Z Time of the mission.



I am happy figured out how to do a 2K recording with my triple screen.]
Here is a 2k Video - even using time compression the issue occurs at the same spot in the flight.



Here is a MSI Afterburner trace showing the issue.
Normally I would see a drop in GPU usage corresponding to the FPS drop.
No really seeing that here - perhaps the load the recording puts on the video card is masking it.
[Linked Image]

I did not post this to extend this discussion.
As others think, I think this is just the way it is.

Regards,



Attached Files 03a Third run.jpg
Last edited by Stache; 06/16/19 09:58 PM.

Insanity: doing the same thing over and over again and expecting different results. A. Einstein

(System Specs:)

I7-6700k OC 4.4GHZ, 16GB DDR4 3200Mhz; Gigabyte Gaming 7 MB, G1 OC'ed GTX980ti; Three-Acer XB271HU WQHD Gsync 144Mhz; Samsung 950-512GB NVMe SSD; WD 2TB-7200rpm; Cooler Master HAF XB EVO, Nepton 240M cooler, V1000 PS; Windows 10 PRO; VKB GunfighterPro Stick; Thrustmaster TPR Pedals; Saitek Throttle; Dual TM MFD panels; TrackIR 5
#4478418 - 06/16/19 10:23 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Nov 2014
Posts: 2,262
Fullofit Online content
Member
Fullofit  Online Content
Member

Joined: Nov 2014
Posts: 2,262
Ajax, ON
Gentlemen, I was comparing your videos and I don't think you are in the same spot. See the image below of the test at more or less same time, when the first Fokker comes into view. Disregarding the sky, you can tell these are two different airfields. Perhaps same fields but in different location? Even trees are different. Is one of you using the airfield mod and the other one isn't? This could explain why they fly to the left while the other to the right. Just a guess.

[Linked Image]

Attached Files Capture.JPG

"Take the cylinder out of my kidneys,
The connecting rod out of my brain, my brain,
From out of my arse take the camshaft,
And assemble the engine again."
#4478426 - 06/16/19 11:41 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2015
Posts: 692
Stache Offline
Member
Stache  Offline
Member

Joined: Sep 2015
Posts: 692
Michigan, USA
Different Dates?

Ad mentioned I am not using any mods.

[Linked Image]

Attached Files flight info (Medium).jpg
Last edited by Stache; 06/16/19 11:45 PM.

Insanity: doing the same thing over and over again and expecting different results. A. Einstein

(System Specs:)

I7-6700k OC 4.4GHZ, 16GB DDR4 3200Mhz; Gigabyte Gaming 7 MB, G1 OC'ed GTX980ti; Three-Acer XB271HU WQHD Gsync 144Mhz; Samsung 950-512GB NVMe SSD; WD 2TB-7200rpm; Cooler Master HAF XB EVO, Nepton 240M cooler, V1000 PS; Windows 10 PRO; VKB GunfighterPro Stick; Thrustmaster TPR Pedals; Saitek Throttle; Dual TM MFD panels; TrackIR 5
#4478434 - 06/17/19 12:54 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
What you failed to mention was the stuttering of the lead planes during take off and the single stutter when passing from the low level to higher level at ~55 seconds during take off. But then, these are normal stutters that I have on all of my WOFF UE games (no matter what PC it is on).


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478436 - 06/17/19 01:05 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Sep 2015
Posts: 692
Stache Offline
Member
Stache  Offline
Member

Joined: Sep 2015
Posts: 692
Michigan, USA
Thanks for the input PR.
I cannot discern anything of note at takeoff.
As for the second - please see note in the afterburner trace.
Regards,

Last edited by Stache; 06/17/19 01:48 AM.

Insanity: doing the same thing over and over again and expecting different results. A. Einstein

(System Specs:)

I7-6700k OC 4.4GHZ, 16GB DDR4 3200Mhz; Gigabyte Gaming 7 MB, G1 OC'ed GTX980ti; Three-Acer XB271HU WQHD Gsync 144Mhz; Samsung 950-512GB NVMe SSD; WD 2TB-7200rpm; Cooler Master HAF XB EVO, Nepton 240M cooler, V1000 PS; Windows 10 PRO; VKB GunfighterPro Stick; Thrustmaster TPR Pedals; Saitek Throttle; Dual TM MFD panels; TrackIR 5
#4478468 - 06/17/19 10:23 AM Re: Need Help With Stutter Testing [Re: kksnowbear]  
Joined: Mar 2006
Posts: 2,239
MajorMagee Online content
Member
MajorMagee  Online Content
Member

Joined: Mar 2006
Posts: 2,239
Dayton, OH
Originally Posted by kksnowbear
Originally Posted by 33lima
HarryH, do you have WoFF on a conventional hard drive or on a Solid State Drive? If your stutter is really related to stuff being loaded, an SSD should help, and they are neither expensive to buy nor difficult to install.

Following an ATAG forum suggestion, I recently eliminated horrible pauses in Cliffs of Dover every time a radio message was being played, by using a Steam function simply to move CloD from the HDD I had it on, to an SSD. Likewise, an SSD has long been known to reduce the regular hiccups you would get playing routes with denser than stock scenery in MS Train Simulator, which loaded the terrain and scenery in quite small chunks as you moved along the line; not seen since I moved to playing MSTS content in Open Rails AND moving it onto an SSD.


Of course, everything you've said about SSDs is correct. And they have been known to work wonders when eliminating this type of stutter from most any other game I can think of.

However - don't get me started about this game and (this specific) stutter - let me tell you this: Not only have I tried (what I call) "stupid fast" storage; that is to say a Samsung 950 Pro NVMe drive with a read speed in excess of 2200 Mbit/s...but I've also gone as far as creating a RAMdrive to load the entire sim to...

...and it makes absolutely. No. Difference. (And we're talking RAM that is easily many, many times faster than even a 'stupid fast' SSD.) Years ago, I tried 4 SSDs in a RAID 0 array (hardware based, not a crappy motherboard software implementation.) Nothing changed.

This game is plagued with (this specific) stutter. There is no solution, IMHO. It has nothing to do with settings, or how much someone spends on hardware, what FPS they claim to have, overclocking (really?), or anything else. CFS3 was known for this issue long ago, and it seems to be going nowhere, in spite of decades of technological improvement in hardware (like SSDs). People who have insisted they have no stutter, but then post videos which show this exact issue...or claiming they never drop below x FPS, but turn around and say they have the same recognizable hitch or glitch or whatever...

I build lots of gaming machines for 'competitive gamers'. Already done several this year. Every one comes with a 100% absolute money-back guarantee, yet no one's ever brought one back. They're out there, every single one of them, playing all sorts of different games, usually in some sort of competitive environment by some very picky players. Many of them overclocked to within an inch of their bloody little silicon lives...

...and no one has any complaints about stutter, or any performance issues at all. I must have some idea what I'm doing. At least they seem to think so...every single one this year so far was a referral from other people I've built for.

Unfortunately, as accurate and well-founded as your mention of faster storage may be...it doesn't solve this issue. TBH, I'm not even sure it improves the issue at all; where, as you pointed out, other games are often improved substantially. Even with the RAMdrive, the stutter I've tested repeatedly is unchanged.

People often don't see the stutter simply because they're not encountering the condition(s) which would trigger the stutter. This is why a repeatable test is important. I tried to do the same thing a long time ago and was blasted for it, by people who just couldn't admit this problem exists. Yet, somehow, oddly enough, it is the one subject that still comes up after all this time, and the one thing that (most) people do acknowledge, and the one thing there's been no solution for (well, maybe the cloud 'popping' issues, as well).

If there are no stutters, then why's it still coming up?

My observations, for whatever it's worth.

HH - best of luck, in all sincerity. But, as you have so accurately observed: A test that can be reliably duplicated is paramount, first and foremost.


Like everything else in CFS3 they made decisions in the coding to limit the impact they would have on computer systems envisioned for 2002 and a few years after. From the behaviours I've observed while trying to push things farther out with compositescenerybudgets.xml, you usually reach a point where the underlying code can't properly manage the scope of the memory required, and it either slows to a crawl, hitches badly, or crashes. This happens in cases where hardware monitoring shows that a modern high end system is not even close to reaching it's processing and memory limits.

For years I've chased after eliminating a repeating stutter with hardware improvements. I eventually attributed the issue to the code needing to flush a fixed memory buffer on a regular basis, and no amount of hardware was going to change that. Only limiting the amount of what was being pushed through that bottleneck could make the hitch happen less frequently.

p.s. It seems to be related to blitting because the speed and direction the screen is scrolling has an effect, with pure left to right at high speed at low altitude (more visible scenery objects to calculate) being the worst.


Service To The Line,
On The Line,
On Time

US Army Ordnance Corps.
#4478479 - 06/17/19 12:04 PM Re: Need Help With Stutter Testing [Re: MajorMagee]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Originally Posted by MajorMagee
Like everything else in CFS3 they made decisions...

MM,

Thank you, thank you, thank you...this has got to be the most lucid explanation I've ever seen of what's going on here, and by far the nearest to completely accurate, far as I'm concerned. Although I'd have to give it some thought, I believe your assessment is probably spot-on.

What's been offensive to me all this time is people saying it was my setup (it's not), or - like you touched on - that their (much more) expensive hardware somehow solved the problem (it doesn't). Just like you said, I realized long ago that the hardware isn't even breaking a sweat to deal with the sim, but these problems are happening anyway.

As for why it was done in the first place, I agree and understand what you mean. My conversations with AnKor have confirmed that these issues are probably due to "the engine" being designed around the way systems worked at the time it was being developed - and things have changed dramatically since then.

Originally Posted by MajorMagee
...and no amount of hardware was going to change that

Precisely. It's just very annoying to me that it's sometimes represented as "My brand new $2000 system doesn't have this issue", when just as you say, no amount of hardware (or money spent thereon) is solving this issue. The problem I have with it is that, for the more impressionable people who are here or wander through and read this, they are convinced to go out and spend a ton of money for newer and newer hardware, when it's not the issue and it's not gong to solve the problem. It's very misleading and inaccurate, and I think it's really doing people a serious wrong when there's a lot of money involved.**

On that note, and out of curiosity...what do you believe is a reasonable system to run this sim, beyond which there is a decided diminishing return? I've often wondered...but it seems to me you could probably do just fine with first-gen 'Core i' CPUs and maybe a 780 GPU? Maybe that's a bit conservative, but man, I can build systems like that for very little...

Your thoughts?

** To be clear: I don't care and it's not my business how much someone spends or can afford to spend on computers. My specific concern is when it's represented that buying new and often very expensive components has somehow changed the fact that this sim is built on an old engine that just cannot benefit in proportion to the cost of top-end hardware.

Last edited by kksnowbear; 06/17/19 12:08 PM.

No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478487 - 06/17/19 12:34 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
From playing this game since OFF3 (more than 10 years ago), I can categorically say the stronger the PC, the smoother the game.

Does it stop the loading "stutters", no, but it does keep the game running at 60 FPS or more which visually makes the game far smoother that if you have a weaker PC that drops the game down into the 40 FPS when things get hot and heavy or down low close to the terrain.

I can definitely tell the difference in smoothness from my top gaming PC (i9 9900K, RTX 208 TI) to my older gaming PC (i7 6700K, GTX 980 Ti).

Are the loading "stutters" there in both PC's, yes, but overall smoothness if far greater in the new PC than the older PC.


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478492 - 06/17/19 12:52 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
The stutters aren't limited to loading, and that's been proven. Panning the view around may (or may not) help, but there will still be stutters later in the sim.

No one's saying faster hardware doesn't run better. I'm saying it doesn't solve the stutters, doesn't guarantee a stable frame rate with no dips, and doesn't add performance to this sim anywhere near in proportion to cost. Not everyone can afford thousands of dollars in new computer hardware every year or so, and if you're going to have dips and hitches anyway ( as you've said yourself) then might as well save a few thousand dollars.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478499 - 06/17/19 01:23 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
There you go again, tell me what something is worth to me and what is not worth something to me !!!!!!!!!!!!!!!!!!!!

I do not tell you what size house to live in, so do not tell me what my PC / game experience is worth to me too.

It my not be worth anything to you, but you have different standards for things versus me, so keep you judgment calls to your self, and not say that others should or should not do when it comes to what they are willing to spend their money on.


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478502 - 06/17/19 01:33 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
I never said anything about you personally. I said "Not everyone can afford thousands of dollars in new computer hardware every year or so..." I'm glad if you have lots of disposable income, but most everyone I know can't afford that.

Stop trying to make something out of nothing.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478503 - 06/17/19 01:46 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
But you are incorrect, strong hardware does stop all the dips, Except for the loading stutters, so it is well worth it to me, even if it is not worth it to you.

And yes, it is something to me when you espouse incorrect opinions about things when you do not even own the game (or have you repurchased it again) ???


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478504 - 06/17/19 01:53 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Originally Posted by Panama Red
Normally anytime you get a "hitch" like that is when the game is "loading" something, be it new planes or new terrain.

I normally get this "hitch" at about 1,000 ft during take-off and then it is fine till it loads some more terrain or squadrons further into the game and I have a pretty decent PC.

You will find, the higher the Scenery Detail or Regional Air Activity you have in the Workshop, the more small "hitches" you will have during the game.

But then this is a 2002 "executable" and AnKor has done wonders just getting us this far with this old CFS3.exe.


Obviously, it happens even later in the game than just loading for you, too.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478506 - 06/17/19 02:18 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
That is correct, it can happen at anytime the game loads new planes or terrain as you are flying, being at the beginning, middle or end of the game.

One way to reduce the various loading stutters is to lower you Regional Air Activity setting, Airfield settings, Woods settings and Detail setting. That way it is not continously loading as many new things as you are flying your mission.

But then everything has a cost, so it is up to you whether reducing these things visually is worth the lessening the occasional loading stutter as you are flying your missions.

As they say, "to each his own".


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478528 - 06/17/19 04:26 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Agreed, but as was discussed above: No amount of settings changes, or cost, will actually eliminate the stutters.

Everything has a cost, yes. The problem is just as Major described it, no amount if hardware fixes this.

Absolutely, to each his own. I simply prefer not to spend 3x as much money on hardware just to wind up with a system playing a game that still stutters. Like I said, if you're going to have stutters anyway, then save a few thousand dollars. I can almost certainly get within 95% of the performance for less than 35% of the cost.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478541 - 06/17/19 06:17 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: May 2001
Posts: 1,651
Panama Red Offline
Member
Panama Red  Offline
Member

Joined: May 2001
Posts: 1,651
Irmo, SC, USA
Isn't that is what is so great about this country and game, you get to make your own choice as to what you think is important and what you think is not important and then go with it. biggrin


CPU = i9 9900K, GPU = RTX 2080 Ti, Monitor = 32" 4K G-sync
#4478555 - 06/17/19 08:22 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Yes sir, and for the record, I spent 10 years defending that very freedom.

I hope you understand it's just strongly against my beliefs to advise anyone to spend a lot of money - especially those who don't have a lot - when they're not really getting that much more for a lot more cost. Diminishing returns, and all that. And that well-known effect is amplified with older games like WOFF/WOTR - they're just not written to take advantage of the hardware, just like Major said. The few problems that remain are just not going to be corrected by even a million-dollar system.

I have never built a $3000 computer. I have built a good many $500-$1000 machines which I'd stack up cost-per-performance against most anything. And, if you're like most I do business with, it doesn't matter how good it is if you can't afford it. Like most guys, I've admired many beautiful high-performance cars. I drive a Toyota pickup, because that's far closer to my reality.

In fact, the biggest reason I do what I do (after just enjoyment of the work) is that I can bring very high performance down to where every day working people with families can possibly afford it. Or maybe a kid who's working part time in retail. That kid needs to save his money for school or helping his family - and that's exactly the advice I give them. Let me build you a machine that will handle most anything you can throw at it, and keep the extra $2000. Haven't had anyone yet object, and they're all very pleased with the outcome. I also offer them a week or two, no-questions-asked return if they just aren't impressed. No one's ever asked me to take one back yet.

Last edited by kksnowbear; 06/17/19 08:44 PM.

No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478792 - 06/19/19 01:34 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Well I am a glutton for punishment, aren't I?

...but as far as I'm concerned, I've finally beaten the infamous stutter challenge. Of course, the competition organizer would quibble and invalidate my video entry, since the AI "isn't set right" or some other argument (he loves to argue). Thankfully I won't know, as I'm so tired of his comments, he's now set to "ignore".

For those that are interested, I do agree with the Major, the stutters come from the fact that it's a very old game engine design. As some of you know I used to be "in the industry" albeit on the soundcard side, but Creative also were one of the first 3D graphics add-in vendors. Back in the late 90's / early 2000's, the developers were always talking about the 3D pipeline and how critical it was to optimize the game engine's communication with the then very limited 3D hardware resources. 3D accelerator cards at that time had a fraction of the dedicated memory that they have today, so games had to be extremely mindful of that in their design.

All that said, I have made some pretty significant improvements to my WOFF performance recently, one coming from the discovery that the roll settings in TrackIR absolutely can mess with things. By fine tuning my TrackIR profile, running G-Sync at half refresh rate, making numerous tweaks in the Inspector, and admittedly by turning down the terrain detail settings (although the challenge organizer never did specify the required detail settings), I've successfully recorded a video that's extremely close to, or actually surpasses the demands of the original test. The final discovery was that, surprisingly enough, the BT mod was NOT helping me! I disabled it. The only way I could get close to the test requirements in terms of the action was to "force" the AI into retreat mode by setting the combat mode to "advantage". Other than that, I think I've more than met the test requirements (and I'm finally mastering the darned Dr1!). You be the judge. If you agree, feel free to give me a thumbsup



..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4478795 - 06/19/19 02:00 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jul 2012
Posts: 1,885
Adger Offline
Member
Adger  Offline
Member

Joined: Jul 2012
Posts: 1,885
Wow how smooth is that,excellent work HH. I don't have Gsync but I'd be more than happy to have a copy of your tweaked trackir profile if that's possible? Yep the BT mod is giving me water texture issues,PR and JJJ are (I think) having a nosy to try and help.
Hope they can get me sorted because I love Triple J,s ME but if I can somehow disable the BT part of it then brilliant.

I wasn't logged in to my YouTube account so I couldn't give you a thumbs up there but here it is ..Great work pal. thumbsup


"Vis unita fortior"

"United Strength is Stronger"
#4478796 - 06/19/19 02:07 AM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jan 2016
Posts: 425
kksnowbear Offline
Member
kksnowbear  Offline
Member

Joined: Jan 2016
Posts: 425
Not even the same test, so it didn't beat anything. The entire area appears differently, and so does the weather. Not even sure this is the same region or date (which were specified). These aren't quibbles, they're called 'criteria', which can't be left out just for the sake of 'winning'.

Continuing to omit more and more of the original challenge, and claiming this means solving problems. I can win the freakin' Boston Marathon, if I simply leave out the first 20 or so miles.

Captain Kirk and the Kobayashi Maru, and just as fictitious. Since I'm the 'organizer' (your words), I'll be sure to note the official records of the test with an asterisk by your "win" (a la Barry Bonds). Great win, champ.


No one has ever successfully completed the Stutter Challenge. Speaks volumes that someone would claim to be a winner because he cheated (Barry Bonds*)

I would prefer even to fail with honor than win by cheating - Sophocles
#4478798 - 06/19/19 02:24 AM Re: Need Help With Stutter Testing [Re: Adger]  
Joined: Dec 2014
Posts: 981
HarryH Offline
Member
HarryH  Offline
Member

Joined: Dec 2014
Posts: 981
Thank you Adger. Stand by for a PM...


..."I took the Stutter Challenge....AND WON!!!"

...”My PC’s been banned... for taking PEDs!!!”

...”The stutters only happen on Wednesdays when it’s cloudy. The rest of the time we’ll be just fine!!!”
#4478898 - 06/19/19 02:13 PM Re: Need Help With Stutter Testing [Re: HarryH]  
Joined: Jul 2012
Posts: 1,885
Adger Offline
Member
Adger  Offline
Member

Joined: Jul 2012
Posts: 1,885
Originally Posted by HarryH
Thank you Adger. Stand by for a PM...



Thanks Harry,much appreciated.


"Vis unita fortior"

"United Strength is Stronger"
Page 1 of 3 1 2 3

Moderated by  Polovski 

Quick Search
Recent Articles
Support SimHQ

If you shop on Amazon use this Amazon link to support SimHQ
.
Social


Recent Topics
Checking in from Spokane our new home
by JimK. 07/18/19 11:45 PM
Old Crow Mustang
by DBond. 07/18/19 08:42 PM
Trailer for you know what!
by archermav. 07/18/19 07:58 PM
Epic Game account hacked? Any suggestions?
by Coot. 07/17/19 09:38 PM
Blink of An Eye - movie trailer
by David Kennard. 07/17/19 08:26 PM
Interesting article on Apollo 11
by oldgrognard. 07/16/19 06:07 PM
Random Thoughts
by vonBaur. 07/16/19 04:49 PM
Renovations
by Alicatt. 07/16/19 10:48 AM
The Darwin Effect
by vonBaur. 07/15/19 04:02 PM
Copyright 1997-2016, SimHQ Inc. All Rights Reserved.

Powered by UBB.threads™ PHP Forum Software 7.6.0