Tribler 5.9.18 pegging CPU

Please create a single thread for each bug report in this forum

Tribler 5.9.18 pegging CPU

Postby 2whomit » Fri Jun 29, 2012 5:43 am

Tribler 5.9.18
swift 5.9.12
Ubuntu 12.04, AMD64, 8GB RAM

The problem reported earlier regarding Tribler 5.9.15 using 100% of CPU has improved a lot with this new version, but is not gone. It just takes longer to happen.

For more background, see the later posts in http://forum.tribler.org/viewtopic.php?f=6&t=2023&p=4315#p4315

I could reproduce this problem consistently when downloading four files, taking up to 18 minutes to peg the CPU. Memory usage gradually climbs, about 2MB per minute, until the CPU pegs, when it climbs drastically. Must end the python process to bring CPU and memory usage down.

I am now doing a test with only one file downloading. It has been over an hour, and CPU has not pegged yet, and memory usage is climbing much more slowly. The problem seems to be dependent on how many files you are downloading.

Also had a problem with swift segfaulting once during these tests; reported in a separate thread.
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby 2whomit » Fri Jun 29, 2012 7:32 am

Have done some more tests. Pausing and resuming various downloads.

Ran one test for a couple hours where I downloaded five files simultaneously and had no problems. Then I added a sixth file, and CPU pegged almost immediately. What was special about that sixth file? It was swift-enabled. The other five were not.

I just realized that all of the CPU-hogging problems I've had with Tribler occurred while downloading swift-enabled torrents. When downloading only torrents that are not swift-enabled, I had no problems.
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby 2whomit » Sat Jun 30, 2012 6:23 am

Have come to the realization that I don't understand how torrents become swift-enabled. So some of my earlier observations may be based on misinterpretations and faulty recollections.

All of my 12 torrents have now finished downloading and are seeding. The five files I mentioned earlier are now marked as swift-enabled, where they weren't before. Out of the 12 total torrents, only one is not marked swift-enabled.

Now that I'm not downloading torrents, I can enable all seeds and run for a long time without pegging the CPU. I'm pretty sure that all previously observed incidents of CPU getting stuck at 100% happened when at least one torrent was still downloading. Not sure now about the relation with swift-enabled.

Memory usage is increasing very gradually. I don't know if this is important.

Also -- I just let Ubuntu upgrade its kernel a couple hours ago. Don't know if this is affecting my results.
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby 2whomit » Sat Jun 30, 2012 8:59 am

I've been doing more tests, trying to understand the behavior I'm seeing.

As I understand it now, torrents aren't marked as swift-enabled until after they have been completely downloaded, and not all of them are so marked. (Correct me if I'm wrong...)

As soon as a torrent is marked as swift-enabled, I have to tell it to Stop Seeding or else bad things happen within a few minutes. (UI becomes sluggish, download&upload speeds start to drop, memory consumption starts to climb more rapidly, and then CPU hits 100% and stays there until the python process is manually ended.) Note that I'm testing with torrents that aren't very popular, and nobody is downloading from my seeds -- but those seeds can bring down Tribler in spite of their lack of bandwidth.

As long as I have told all swift-enabled seeds to stop seeding, I can add a bunch of new torrents and download for a while with no problems. However, If I add too many simultaneous downloads, I can still overwhelm Tribler, and then have to force quit it. But I don't have to manually end the python process in this case.

So it seems clear that Tribler is much more sensitive to swift-enabled seeds, even if there's only one that's active and nobody is touching it, than it is to a bunch of big downloads.

Given all of that, I suspect that there is a problem with swift. (The single segfault I got from swift a couple days ago would seem to confirm this.)
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby 2whomit » Sat Jun 30, 2012 9:19 pm

Just ran another test. Let it run for several hours unattended. Six big files downloading, one non-swift file seeding.

When I came back, CPU was at 100% and bandwidth was low (but not zero; clearly Tribler was sending and receiving a small amount of data, less than 15kBps, and after I closed Tribler, bandwidth dropped to less than 1kBps).

Judging by the amount of data downloaded during the test, it had successfully run for at least a few hours at close to maximum bandwidth before it choked up. I consider Tribler quite usable if it can do that.

On the other hand, allowing a single swift-enabled torrent to seed, in addition to the files mentioned above, will bring Tribler to its knees in minutes.
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby Niels » Sun Jul 01, 2012 11:49 am

This really helps, we'll look into swift <-> tribler to see if there are any problems there.
My guess is the DHT integration we provide to swift. After swift starts to seed it will request peers from tribler in order to upload etc. Maybe there is an error there.

Downloads get to be swift-enabled after they have completely downloaded and are a complete swarm, i.e. you are downloading all files from a swarm not deselecting any file.
Niels
Addict
 
Posts: 718
Joined: Mon Aug 23, 2010 1:56 pm

Re: Tribler 5.9.18 pegging CPU

Postby 2whomit » Mon Jul 02, 2012 9:34 pm

Looking back over all the posts I've made regarding this issue, it now seems that all the guesses I have made about the patterns behind these problems have been wrong in some way. I cannot really discern much consistency in Tribler's behavior. One time it seems like one thing is triggering the problem, then another time it's some other thing.

This may be happening because I cannot see a bunch of the data. The Downloads screen only shows what's happening with torrents I've actually told Tribler about (and even then, the displayed data is sometimes terribly stale, not updated at all for many seconds). But behind the scenes, lots of other things are happening, like all those unrelated .torrent files being swapped back and forth between peers. I see no stats on those activities.

Therefore, when I look at System Monitor, it shows bandwidth being consumed in both directions that I cannot account for. There often seems to be little correlation between the totals reported by Tribler and those reported by System Monitor. This leads me to mistrust the numbers I see, and suspect the presence of bugs anywhere and everywhere. Patterns become harder to discern. Perhaps I could make more sense of what I am seeing if the downloads page also showed stats for those other activities. And, it would certainly be easier if all the stats were displayed on a more consistent schedule -- perhaps have a setting for display interval, defaulting to one second, but I could change it to longer intervals to see if that affected the CPU choking?

Furthermore, I cannot confidently do experiments with bandwidth limits within Tribler. I sometimes try to do experiments involving raising or lowering those limits, and seeing what effect this has on the CPU choking problem. But half the time it seems like those limits are having no effect, i.e. I set the download limit to 60KBps, and often both Tribler and System Monitor will tell me that my downloads are way beyond that limit! I am left scratching my head, wondering if the reported bandwidth numbers are wrong, or if the bandwidth limits simply don't work, or possibly both.

(I know, these are wishlist items and not bug reports... but at least they are related to improved bug reporting... I think...)
2whomit
Visitor
 
Posts: 32
Joined: Wed Jun 20, 2012 5:17 am

Re: Tribler 5.9.18 pegging CPU

Postby schaap » Tue Jul 03, 2012 9:04 am

Thanks for all your testing, 2whomit. It's worth a lot to see these kind of problems arise. The patterns may be hard to discern for you - that's not really a surprise given the complexity of Tribler - and your guesses in fact contain important pointers for us. Some of the developers here can figure out what's happening exactly using your guesses and other data.

Speed limits in Tribler are currently per-torrent only. So there is no general speed limit on all traffic, which explains why you see total speeds beyond your set limits. I agree that having more consistent, or rather, complete data reporting would be a good thing. It's not strange that people suspect bugs if the numbers don't add up. Even if we'd just report everything besides actual data transfers as 'overhead'.

I doubt changing the interval on the statistics updates is going to do anything for your CPU usage: it's a simple interface update using some already known values. For experiments under great stress you only start taking that into account when the interval is below, say, 10 ms. I'm not sure on the actual value, but it's safe.
schaap
Visitor
 
Posts: 130
Joined: Tue Jan 03, 2012 4:09 pm


Return to Bug Reports

Who is online

Users browsing this forum: ugozz and 1 guest

cron