Spark
  1. Spark
  2. SPARK-1363

decrease ChatTranscriptPlugin loading time and Ram

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Product Backlog
    • Component/s: None
    • Labels:
      None

      Description

      need to speedup history loading
      and remove the 5000message limitation

        Activity

        Hide
        Walter Ebeling added a comment -

        no fix available for 2.6.1

        Show
        Walter Ebeling added a comment - no fix available for 2.6.1
        Hide
        Mike McMahon added a comment -

        I found a marked increase if we switch to an InputStream / BufferedInputStream vs. a File when reading in the transcripts. The read time went from 500ms average to about 100ms average, this is just on loading a large chat file, not displaying. Still need to find a way to speed up displaying of large chat logs

        Show
        Mike McMahon added a comment - I found a marked increase if we switch to an InputStream / BufferedInputStream vs. a File when reading in the transcripts. The read time went from 500ms average to about 100ms average, this is just on loading a large chat file, not displaying. Still need to find a way to speed up displaying of large chat logs
        Hide
        Mike McMahon added a comment -

        Also found that by calling window.setPage vs. window.setText we save (on my system) roughly 5-6 seconds on average. The only caveat is that to use setPage we need to take the chat log that has been parsed to HTML and save it as an HTML file somewhere and reload it in as a URL and pass that url into the window.setPage method.

        Here are my timings from two runs (the first being with setPage and the second with setText)
        Getting transcript stream forwroot@igniterealtime.org
        Start xml parsing - 1310793455403
        Finish xml parsing - 1310793456032
        Parsing Start - 1310793456109
        Parsing finished - 1310793456452
        Setting the text - 1310793456452
        Running a replace on the builder... - 1310793456508
        Ending - 1310793456509
        1106

        Getting transcript stream forwroot@igniterealtime.org
        Start xml parsing - 1310793626502
        Finish xml parsing - 1310793627004
        Parsing Start - 1310793627121
        Parsing finished - 1310793627427
        Setting the text - 1310793627428
        Running a replace on the builder... - 1310793634475
        Ending - 1310793634475
        7973

        As you can see it is a lot of time saved! The major diff. being that setPage calls updates/draws once the entire file has been loaded whereas setText redraws the handle after each line (which is why very large files can take quite a bit of time to load).

        You can definitely feel the setPage drawing though - it may take about 1 second to finish loading the large file (in my case it's roughly 3MB) but it takes an additional one or two seconds to finish the update/draw method on that much text. I'm certain there are more ways to speed this up though.

        Show
        Mike McMahon added a comment - Also found that by calling window.setPage vs. window.setText we save (on my system) roughly 5-6 seconds on average. The only caveat is that to use setPage we need to take the chat log that has been parsed to HTML and save it as an HTML file somewhere and reload it in as a URL and pass that url into the window.setPage method. Here are my timings from two runs (the first being with setPage and the second with setText) Getting transcript stream forwroot@igniterealtime.org Start xml parsing - 1310793455403 Finish xml parsing - 1310793456032 Parsing Start - 1310793456109 Parsing finished - 1310793456452 Setting the text - 1310793456452 Running a replace on the builder... - 1310793456508 Ending - 1310793456509 1106 Getting transcript stream forwroot@igniterealtime.org Start xml parsing - 1310793626502 Finish xml parsing - 1310793627004 Parsing Start - 1310793627121 Parsing finished - 1310793627427 Setting the text - 1310793627428 Running a replace on the builder... - 1310793634475 Ending - 1310793634475 7973 As you can see it is a lot of time saved! The major diff. being that setPage calls updates/draws once the entire file has been loaded whereas setText redraws the handle after each line (which is why very large files can take quite a bit of time to load). You can definitely feel the setPage drawing though - it may take about 1 second to finish loading the large file (in my case it's roughly 3MB) but it takes an additional one or two seconds to finish the update/draw method on that much text. I'm certain there are more ways to speed this up though.
        Hide
        wroot added a comment -

        Any patch to test that? Also, maybe this is related to SPARK-1404?

        Show
        wroot added a comment - Any patch to test that? Also, maybe this is related to SPARK-1404 ?
        Hide
        Mike McMahon added a comment -

        sure, it's a very messy patch so let me get it together, definitely not what i would want to see in the source

        Show
        Mike McMahon added a comment - sure, it's a very messy patch so let me get it together, definitely not what i would want to see in the source
        Hide
        Mike McMahon added a comment -

        Pretty messy, not what i would put in the source.

        Basically we can speed up the portion of loading/dumping into the JEditorPane to under a second (on certain systems) but it's the drawing on the JEditorPane that's killing us. You can (in the code) change the window.setText() / window.setPage() (line 473/474 in ChatTranscriptPlugin) to see how it affects the performances (on the console output).

        It becomes painfully apparent where our slowness is when you use the setPage() method!

        Show
        Mike McMahon added a comment - Pretty messy, not what i would put in the source. Basically we can speed up the portion of loading/dumping into the JEditorPane to under a second (on certain systems) but it's the drawing on the JEditorPane that's killing us. You can (in the code) change the window.setText() / window.setPage() (line 473/474 in ChatTranscriptPlugin) to see how it affects the performances (on the console output). It becomes painfully apparent where our slowness is when you use the setPage() method!
        Hide
        wroot added a comment -

        I didn't notice much improvement on huge history loading. It takes same amount of time. Maybe a scrollbar appears faster, but it is still not possible to use it for some time.

        Show
        wroot added a comment - I didn't notice much improvement on huge history loading. It takes same amount of time. Maybe a scrollbar appears faster, but it is still not possible to use it for some time.
        Hide
        Mike McMahon added a comment -

        Yeah it's the drawing that's killing us unfortunately. the setPage loads faster but once it calls to draw it takes roughly the same amount of time.

        Show
        Mike McMahon added a comment - Yeah it's the drawing that's killing us unfortunately. the setPage loads faster but once it calls to draw it takes roughly the same amount of time.
        Hide
        Mike McMahon added a comment -

        please test the newest patch, i switch to using invokeLater for loading the text vs. nothing at all.

        This seems to cut down on the loading by about half on my system. Though i would love to get some more input from others.

        Show
        Mike McMahon added a comment - please test the newest patch, i switch to using invokeLater for loading the text vs. nothing at all. This seems to cut down on the loading by about half on my system. Though i would love to get some more input from others.

          People

          • Assignee:
            Unassigned
            Reporter:
            Wolf P.
          • Votes:
            2 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated: