Skip to content
Snippets Groups Projects
  1. Jan 29, 2024
  2. Jan 25, 2024
  3. Jan 23, 2024
  4. Jan 20, 2024
  5. Jan 19, 2024
    • Joey Hess's avatar
      1f899661
    • Joey Hess's avatar
    • Joey Hess's avatar
      add directional stalldetection and bwlimit configs · 20567e60
      Joey Hess authored
      Sponsored-by: Dartmouth College's DANDI project
      20567e60
    • Joey Hess's avatar
      use watchFileSize in Remote.External.retrieveKeyFile · c02df792
      Joey Hess authored
      external: Monitor file size when getting content from external special
      remotes and use that to update the progress meter, in case the external
      special remote program does not report progress.
      
      This relies on 703a70ca to prevent ever
      running the meter backwards.
      
      Sponsored-by: Dartmouth College's DANDI project
      c02df792
    • Joey Hess's avatar
      RawFilePath conversion · 8da85fd3
      Joey Hess authored
      Sponsored-by: Dartmouth College's DANDI project
      8da85fd3
    • Joey Hess's avatar
      avoid watchFileSize running backward · 703a70ca
      Joey Hess authored
      This is groundwork for using watchFileSize for downloads from external
      special remotes.
      
      In Annex.Content.downloadUrl, this potentially avoids jitter in the
      progress meter. When downloading with conduit, the meter gets updated based
      on both the size of the file, and on the data flowing through conduit.
      If that has not yet been flushed to the file, it seems possible for the
      meter to run backwards when meter is updated with the file size.
      It's probably only a few kb of jitter, so may not be visible.
      
      Sponsored-by: Dartmouth College's DANDI project
      703a70ca
    • jstritch's avatar
      Added a comment · 4276cb20
      jstritch authored
      4276cb20
    • Joey Hess's avatar
    • Joey Hess's avatar
      tweak stall detection scaling · df35f708
      Joey Hess authored
      Refactored to allow offline experimentation, and ended up changing the
      allowedvariation (aka fudge factor) to 3. 10 seems too high, and 1.5 too low.
      
      Scale earlier, so even if the first chunk takes less than the configured
      time period, allowance is made that later chunks might transfer slower.
      Decided to use the same allowedvariation to decide when to start
      scaling.
      
      Smoothed the scaling out.
      
      Some examples:
      
      ghci> upscale (BwRate 10 (Duration 60)) 25
      BwRate 13 (Duration {durationSeconds = 75})
      -- A small scaling upwards after 1/3rd the time. Not noticable.
      ghci> upscale (BwRate 10 (Duration 60)) 60
      BwRate 30 (Duration {durationSeconds = 180})
      -- At the configured time, 3x scaling.
      ghci> upscale (BwRate 10 (Duration 60)) 120
      BwRate 60 (Duration {durationSeconds = 360})
      -- A typical upscaling, here a 1 minute duration became 6 minutes
      -- due to the first chunk taking 2 minutes to transfer.
      ghci> upscale (BwRate 10 (Duration 60)) 600
      BwRate 300 (Duration {durationSeconds = 1800})
      -- Here the first chunk took 10 minutes to transfer, so it will
      -- take 30 minutes to detect a stall.
      
      Sponsored-by: Dartmouth College's DANDI project
      df35f708
  6. Jan 18, 2024
    • imlew's avatar
      Added a comment · 0270f4d8
      imlew authored
      0270f4d8
    • Yaroslav Halchenko's avatar
      Added a comment · 3e047f05
      Yaroslav Halchenko authored
      3e047f05
    • Joey Hess's avatar
      comment · e61af28a
      Joey Hess authored
      e61af28a
    • Joey Hess's avatar
      comment · 3ef766c4
      Joey Hess authored
      3ef766c4
    • Joey Hess's avatar
    • Joey Hess's avatar
      automatically adjust stall detection period · c2634e7d
      Joey Hess authored
      Improve annex.stalldetection to handle remotes that update progress less
      frequently than the configured time period.
      
      In particular, this makes remotes that don't report progress but are
      chunked work when transferring a single chunk takes longer than the
      specified time period.
      
      Any remotes that just have very low update granulatity would also be
      handled by this.
      
      The change to Remote.Helper.Chunked avoids an extra progress update when
      resuming an interrupted upload. In that case, the code saw first Nothing
      and then Just the already transferred number of bytes, which defeated this
      new heuristic. This change will mean that, when resuming an interrupted
      upload to a chunked remote that does not do its own progress reporting, the
      progress display does not start out displaying the amount sent so far,
      until after the first chunk is sent. This behavior change does not seem
      like a major problem.
      
      About the scalefudgefactor, it seems reasonable to expect subsequent chunks
      to take no more than 1.5 times as long as the first chunk to transfer.
      Could set it to 1, but then any chunk taking a little longer would be
      treated as a stall. 2 also seems a likely value. Even 10 might be fine?
      
      Sponsored-by: Dartmouth College's DANDI project
      c2634e7d
    • Yaroslav Halchenko's avatar
      Added a comment · 4fe3e595
      Yaroslav Halchenko authored
      4fe3e595
    • Joey Hess's avatar
      improve annex.stalldetection documentation · 8f655f79
      Joey Hess authored
      8f655f79
    • imlew's avatar
      No commit message · cd986d61
      imlew authored
      No commit message
      cd986d61
    • Joey Hess's avatar
      comment · 931920c4
      Joey Hess authored
      931920c4
    • Joey Hess's avatar
      open todo · 3ca1e036
      Joey Hess authored
      3ca1e036
    • Joey Hess's avatar
      comment · ae4177ec
      Joey Hess authored
      ae4177ec
    • Joey Hess's avatar
      document that --listen specifies an IP address · 52d4f36b
      Joey Hess authored
      Avoid users getting confused and thinking this allows specifying the
      port.
      
      Sponsored-by: the NIH-funded NICEMAN (ReproNim TR&D3) project
      52d4f36b
    • Joey Hess's avatar
      cleanup · adb8b320
      Joey Hess authored
      adb8b320