1. 20 Oct, 2012 5 commits
    • Timothy B. Terriberry's avatar
      Re-do abstract stream reader API. · e2d7b266
      Timothy B. Terriberry authored
      This changes op_read_func to
      a) Take a single byte count to read instead of an "item" count
          (which the http backend couldn't properly support anyway).
      b) Use integers for buffer sizes to avoid having to worry about
          sign differences and whether size_t is larger or smaller than
          opus_int64, etc.
      c) Return an explicit error code (instead of using errno like
          fread).
         We had already eliminated the use of errno, but we did it by
          treating read errors and EOF identically in all cases.
         This was preventing us from reporting SSL truncation attacks
          from the https backend.
         The https backend now properly reports such errors.
      
      This commit also fixes a bug introduced in 9b57b0c2, where we
       accidentally started passing absolute offsets to the _boundary
       parameter of op_get_next_page() instead of relative offsets.
      We now use absolute offsets in all places, as it is the simpler
       choice.
      This matters now, because the error reported when encountering EOF
       before hitting the _boundary is no longer suppressed (but instead
       reported as OP_EBADLINK).
      
      Finally, it removes the op_page_seek() function.
      Except for the time needed to decode forward after seeking, this
       function was identical in performance to op_pcm_seek(), and Opus
       requires decoding 80 ms of data after seek anyway, so the relative
       benefit is much smaller than with Vorbis.
      A survey of open-source code using libvorbisfile showed that the
       only usages of ov_page_seek() in the wild were calling it to seek
       to the start of the stream, for which op_pcm_seek() already has a
       special case that makes it just as fast.
      
      The documentation was also updated to describe all of these chanes.
      
      This is an incompatible API change.
      e2d7b266
    • Timothy B. Terriberry's avatar
      Make the URL API more extensible. · 800be8c0
      Timothy B. Terriberry authored
      Right now we have no way to add any more parameters beyond a set of
       basic binary flags.
      This unifies op_url_stream_create() and
       op_url_stream_create_with_proxy() into a single function that
       takes a variable-length list of arguments, which can be extended
       in the future to include more options of any type.
      
      This is an incompatible API change.
      800be8c0
    • Timothy B. Terriberry's avatar
      Fix op_http_parse_connection(). · 756c4c0b
      Timothy B. Terriberry authored
      We were computing the return value correctly, but then not
       returning it.
      756c4c0b
    • Timothy B. Terriberry's avatar
      Some http robustness improvements. · f83266d9
      Timothy B. Terriberry authored
      - Increase the maximum response header buffer size up to ~32 kB.
        This also moves it into a heap-allocated buffer instead of the
         stack, as 32 kB is really too much for the stack.
      - Treat LF as CR LF when parsing headers.
        This is necessary when parsing the load-balancer response in
         front of <http://lazaradio.hu:8100/bermuda.opus>.
        The response returned by that server is invalid in lots of ways,
         but with these two changes we can read it.
      - In addition, we now peek ahead at a large chunk of data when
         reading the response instead of reading 2 to 4 bytes at a time.
        This allows a typical response to be read with two syscalls
         (one peek, one read) instead of several hundred.
      - Stop trying to read more data when the connection is closed.
      f83266d9
    • Timothy B. Terriberry's avatar
      Documentation revisions and a leak fix. · 2ffd8cb7
      Timothy B. Terriberry authored
      Thanks to Jean-Marc Valin for providing documentation feedback.
      This also changes op_tags() to allow the tags to be queried on a
       partially-open file (because they've already been read and are
       available), and fixes a leak when a partially open file is freed
       instead of being fully opened.
      2ffd8cb7
  2. 14 Oct, 2012 1 commit
    • Timothy B. Terriberry's avatar
      Fix some socket connection bugs. · bb8cb7a3
      Timothy B. Terriberry authored
      The big one was that if the connect() call failed, it would loop
       forever (thanks to some code re-factoring, the loop was no longer
       advancing to the next address as originally designed).
      bb8cb7a3
  3. 13 Oct, 2012 1 commit
    • Timothy B. Terriberry's avatar
      Remove an assert. · e88aa498
      Timothy B. Terriberry authored
      On the first request made by a connection, the value it was testing
       might be uninitialized, so we can't guarantee it'll pass.
      e88aa498
  4. 12 Oct, 2012 1 commit
    • Timothy B. Terriberry's avatar
      Major updates to the http backend. · 3d78feff
      Timothy B. Terriberry authored
      * Now supports HTTP/1.1 persistent connections with pipelining.
        This speeds up chain enumeration on large files by almost a
         factor of 2 over http, and by roughly a factor of 4 over https.
        The difference between http and https is now much smaller.
      * Add timeouts to all the socket I/O.
      * Estimate the number of available bytes to read and use it when
         making connection re-use decisions.
      * Add support for https with proxies using HTTP/1.1 CONNECT
         tunnels.
      * Fix TLS session re-use (it requires clean shutdown).
      * Various other code re-organization and minor improvements.
      3d78feff
  5. 09 Oct, 2012 2 commits
  6. 02 Oct, 2012 1 commit
  7. 01 Oct, 2012 5 commits
    • Timothy B. Terriberry's avatar
      Multiple small fixes. · 6c56a973
      Timothy B. Terriberry authored
      * Tags were being freed even if not initialized if
         op_fetch_headers() failed on the first link.
        The logic for handling completely empty links would leak, also.
      * Ignore fragment identifiers in http URLs instead of rejecting
         them.
      * Get the current link in seeking_example's verify_seek when
         op_read_native() fails to return it for us.
      6c56a973
    • Timothy B. Terriberry's avatar
      Fix scan_build static analysis reports. · 737cbf57
      Timothy B. Terriberry authored
      Most of these were dead code left around from rewriting things and
       things llvm doesn't have enough global information to prove by
       itself.
      The one real error was the missing NULL check in
       opus_tags_parse_impl().
      737cbf57
    • Timothy B. Terriberry's avatar
      Fix an uninitialized warning from cf594de6. · 1f2f0bc4
      Timothy B. Terriberry authored
      The warning was correct, as far as it goes, but the use was
       actually harmless.
      1f2f0bc4
    • Timothy B. Terriberry's avatar
      Fix file: <host> comparison. · 01012058
      Timothy B. Terriberry authored
      The previous comparison would have succeeded on things like
       "localhost123" instead of just "localhost".
      01012058
    • Timothy B. Terriberry's avatar
      Clean up page seeking a bit. · cf594de6
      Timothy B. Terriberry authored
      * Guarantee pcm_start and pcm_end stay in range (not just move in
         the right direction).
      * When we fail to find a page in the interval, back up by increasing
         chunk sizes just like op_get_prev_page_serial().
      * Eliminate the special case for the last page in the interval.
      * Force a straight bisection if we're backing up, but not decreasing
         the interval size rapidly enough, to limit the worst-case.
        This is guaranteed not to affect the first two iterations, so it
         has minimal impact on seeking in the normal case.
      cf594de6
  8. 29 Sep, 2012 2 commits
    • Timothy B. Terriberry's avatar
      Fix handling of holes. · e7661191
      Timothy B. Terriberry authored
      We have to call op_collect_audio_packets() again to drain any
       buffered packets from the page.
      e7661191
    • Timothy B. Terriberry's avatar
      Clean-up for alternate configurations. · 21322357
      Timothy B. Terriberry authored
      * s/op_read_stereo_float/op_read_float_stereo/ for the fixed-point
         API.
      * Fix compiler warnings exposed when optimizations are enabled.
      * Fix opusfile_example to work with --enable-fixed-point
         --disable-float
      * Fix seeking_example to not re-define OP_FIXED_POINT if it's
         already been defined.
      21322357
  9. 24 Sep, 2012 2 commits
  10. 23 Sep, 2012 11 commits
  11. 22 Sep, 2012 6 commits
    • Timothy B. Terriberry's avatar
      Save initial stream state on seekable open. · 2df5f786
      Timothy B. Terriberry authored
      This means that in the normal case, after identifying the chain
       boundaries, we will seek back exactly where we were before.
      With the http backend, that gives us a good chance of re-using our
       initial connection to continue reading the stream.
      Currently the LRU connection expiration algorithm means that is
       unlikely with lots of links, however.
      2df5f786
    • Timothy B. Terriberry's avatar
      Small speed-up to op_bisect_forward_serialno(). · 167822cf
      Timothy B. Terriberry authored
      Try to guess that the next link will be approximately the average
       size of all previous links, for files with many links.
      This cuts off 6-17% of the seeks.
      
      Also remove a variable that was left unused after 5e36109d.
      167822cf
    • Timothy B. Terriberry's avatar
      Clean up offset tracking. · 5e36109d
      Timothy B. Terriberry authored
      Reduce the number of places we modify 'offset' so that
       op_seek_helper() can always skip seeks to the current offset.
      The checks we were doing before already covered all the places
       where this was useful in the normal case, but this lets us
       centralize that logic.
      
      This commit also includes a few minor follow-ups to 9b57b0c2:
      * Use a smaller type for ret_size and initialize it.
      * Verify 'end' is at least as large as data we've already read.
      5e36109d
    • Timothy B. Terriberry's avatar
      Fix some issues with trailing junk in files. · 9b57b0c2
      Timothy B. Terriberry authored
      1) We were treating EOF in op_get_next_page() as a read error when
          called from op_get_prev_page_serial().
      2) We also assumed op_get_prev_page_serial() stopped scanning at the
          end of the page it returned, in order to compute the size of
          that page.
         Return the page size explicitly instead.
      3) Finally, once we discover where the last page is, there is no
          reason to ever look at data past it.
         Update 'end' once we find it, and always pass that to
          op_get_next_page().
      9b57b0c2
    • Timothy B. Terriberry's avatar
      Minor http redirect clean-ups. · 073fe98c
      Timothy B. Terriberry authored
      Actually close the old socket.
      Don't initialize SSL more than once.
      073fe98c
    • Timothy B. Terriberry's avatar
      Add support for http, https, and file URLs. · 8d04070b
      Timothy B. Terriberry authored
      This is pretty preliminary for now.
      Seeking performance isn't great, and there's no caching.
      I'm sure there's also lots of bugs.
      8d04070b
  12. 19 Sep, 2012 1 commit
    • Timothy B. Terriberry's avatar
      Fix hole reporting discrepancy. · 78e5fffc
      Timothy B. Terriberry authored
      op_find_initial_pcm_offset() would swallow any holes at the start
       of a link, but seekable streams did not use it when crossing a
       link boundary, so they would report a hole.
      This unifies the hole reporting between the two, and also
       simplifies the seeking code (which also needs to ignore holes) a
       bit.
      78e5fffc
  13. 18 Sep, 2012 1 commit
  14. 17 Sep, 2012 1 commit
    • Ralph Giles's avatar
      Move opusfile.h up a level. · 3f18c55f
      Ralph Giles authored
      This matches the convention used by opus itself, where
      headers are included without a prefix, and the build
      must do -I/path/to/opus to find the appropriate versions.
      3f18c55f