OS:

Android 5.1 100%

Devices:

Archos 36% condor 27% Meizu 23% HUAWEI 14%

Stacktrace:

Fatal Exception: java.lang.ArrayIndexOutOfBoundsException: length=1840; index=320371712
       at okio.Base64.encode(Base64.java:137)
       at okio.Base64.encode(Base64.java:125)
       at okio.ByteString.base64(ByteString.java:131)
       at okhttp3.Cache$Entry.writeCertList(Cache.java:683)
       at okhttp3.Cache$Entry.writeTo(Cache.java:646)
       at okhttp3.Cache.put(Cache.java:250)
       at okhttp3.Cache$1.put(Cache.java:149)
       at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:131)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
       at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
       at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
       at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:143)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
       at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
       at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
       at okhttp3.RealCall$AsyncCall.execute(RealCall.java:147)
       at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:818)

Many Picasso-related tasks in the other threads.

Can provide further info if needed

Took me a while to figure out what was the cause as the issue does not occurs on my CI but updating Okio from 2.0.0 to 2.1.0 cause proguard 6.0.3 on my windows dev machine to goes nuts.

I fear you won't have any idea and I have no idea how to debug that but with Okio 2.1.0 + Moshi 1.7 (And 1.6) + Okhttp 3.11 + some basic okio code in kotlin in my code like

response.body()?.use { body ->
                    return ByteArrayOutputStream().run {
                        this.sink().buffer().use { it.writeAll(body.source()) }
                        toByteArray()?.also { close() }
                    }
                }

Cause proguard to be stuck for infinite time at Optimizing (pass 1/5)... step. Gradle debug mode does not show any issue during that time, just the usual memory checks without any memory issues.

Reverting to 2.0.0 fix the issue.

R8 from AGP 3.2 have an issue I reported that was unfortunately not fixed in time, and R8 from 3.3a12 does compile but Moshi is no more working (was previously when I helped the Google team fixing the R8 issue) and I have not yet found (searched) the cause but maybe it's tied. (Moshi generates {} instead of correct objects).

Edit: Tested 2.0.0 with R8 AGP 3.3a12 and same json issue so proguard issue not related to R8 issue that I'll investigate later.

I have a need for hashing algorithms available on ByteString outside of the JVM. Specifically, the BitTorrent protocol uses SHA-1 to verify that file pieces contain the correct data. Right now, it seems that hashing relies on the JVM MessageDigest implementation.

As a proof of concept, I have an implementation of SHA-1 implemented on my own version of ByteString in my project Corbit. You can see it here.

Our users are running into problems with java.nio.file.Path. We have proguard rules that help to work around this. We should document 'em.

See also https://github.com/square/okio/issues/42

10-06 12:03:39.450 1327 2243 W System.err: java.lang.NullPointerException: Attempt to read from field 'int com.android.okio.Segment.limit' on a null object reference 10-06 12:03:39.460 1327 2261 E AndroidRuntime: FATAL EXCEPTION: Thread-324 10-06 12:03:39.460 1327 2261 E AndroidRuntime: java.lang.NullPointerException: Attempt to read from field 'int com.android.okio.Segment.limit' on a null object reference 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okio.OkBuffer.write(OkBuffer.java:574) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okio.OkBuffer.read(OkBuffer.java:610) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okio.RealBufferedSource.read(RealBufferedSource.java:56) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.http.HttpConnection$FixedLengthSource.read(HttpConnection.java:465) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.Util.skipAll(Util.java:227) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.http.HttpConnection.discard(HttpConnection.java:235) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.http.HttpConnection$FixedLengthSource.close(HttpConnection.java:487) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.Util.closeQuietly(Util.java:97) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.http.HttpEngine.close(HttpEngine.java:574) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.android.okhttp.internal.http.HttpURLConnectionImpl.disconnect(HttpURLConnectionImpl.java:113) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.touchtype_fluency.util.HttpDownload.b(Unknown Source) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.touchtype_fluency.util.HttpDownload.interrupt(Unknown Source) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at com.touchtype_fluency.util.Downloader$2.run(Unknown Source) 10-06 12:03:39.460 1327 2261 E AndroidRuntime: at java.lang.Thread.run(Thread.java:818) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okio.OkBuffer.write(OkBuffer.java:574) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okio.OkBuffer.read(OkBuffer.java:610) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okio.RealBufferedSource.read(RealBufferedSource.java:56) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okhttp.internal.http.HttpConnection$FixedLengthSource.read(HttpConnection.java:465) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okhttp.internal.Util.skipAll(Util.java:227) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okhttp.internal.http.HttpConnection.discard(HttpConnection.java:235) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okhttp.internal.http.HttpConnection$FixedLengthSource.close(HttpConnection.java:487) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okio.RealBufferedSource.close(RealBufferedSource.java:204) 10-06 12:03:39.460 1327 2243 W System.err: at com.android.okio.RealBufferedSource$1.close(RealBufferedSource.java:187) 10-06 12:03:39.460 1327 2243 W System.err: at java.io.BufferedInputStream.close(BufferedInputStream.java:141) 10-06 12:03:39.460 1327 2243 W System.err: at org.apache.commons.io.input.ProxyInputStream.close(ProxyInputStream.java:102) 10-06 12:03:39.460 1327 2243 W System.err: at java.io.FilterInputStream.close(FilterInputStream.java:64) 10-06 12:03:39.460 1327 2243 W System.err: at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:178) 10-06 12:03:39.460 1327 2243 W System.err: at com.touchtype_fluency.util.HttpDownload.downloadZip(Unknown Source) 10-06 12:03:39.460 1327 2243 W System.err: at com.touchtype_fluency.util.b$1.onDownload(Unknown Source) 10-06 12:03:39.460 1327 2243 W System.err: at com.touchtype_fluency.util.Downloader$1.run(Unknown Source) 10-06 12:03:39.460 1327 2243 W System.err: at java.lang.Thread.run(Thread.java:818)

This PR is currently a work in progress to be developed out in the open.

Tasks to be done:

• make a nonJvmMain sourceSet for this to reside in
• make typeAlias of MessageDigest in JVM module
• implement SHA-1 algorithm
• add tests

Relevant: #740

Hello,

I see a strange picture in a running jcmd Thread.print on my server app using okio 2.5.0, many threads are blocking during parallel decoding:

   java.lang.Thread.State: BLOCKED (on object monitor)
     at okio.SegmentPool.recycle(SegmentPool.kt:64)
    - waiting to lock <0x00000002cf4f3c78> (a okio.SegmentPool)

In source code for SegmentPool there are @ThreadLocal and synchronized(this) at the same time. So, how does that pool intended to work?

This is a tracking bug for all the work to implement Okio’s new multiplatform files API.

• [x] Path.name
• [x] Path.parent
• [x] Filesystem.cwd() - current working directory
• [x] Filesystem.list(Path) - list directory contents
• [x] Filesystem.source(Path) - read a file
• [x] Filesystem.sink(Path) - write a file
• [x] Filesystem.atomicMove(Path, Path) - move a file
• [x] Filesystem.copy(Path, Path) - move a file
• [x] Filesystem.delete(Path) - remove a file
• [x] Filesystem.mkdir(Path) - create a directory
• [x] Change empty-string relative path to a single dot? (more conventional)
• [x] Use object pinning with Buffers
• [x] Support Windows
• [x] Something for fsync?
• [x] FakeFilesystem, a first-party in-memory implementation
• [x] Support Linux
• [x] FileMetadata
• [x] Support Windows-like semantics for FakeFileSystem
• [x] Implement and test inode-like semantics for FakeFileSystem
• [x] Support Windows Paths
• [x] Support Windows paths like C:\foo.txt in FakeFileSystem
• [x] appendingSink()
• [x] Support iOS
• [x] Explore options for Node.js. Might motivate one-shot suspend functions to load an entire file as a ByteString?
• [x] JVM methods for interop with java.io.File, java.nio.file.Path
• [x] Support Android releases that don’t have java.nio.file
• [x] Don’t depend on kotlinx-nodejs
• [x] Add an experimental annotation
• [x] Filesystem.deleteContents, to recursively delete a directory
• [x] Filesystem.createDirectories, to recursively create a path of directories
• [x] Filesystem.exists, or some mechanism to differentiate between ‘absent’ and ‘inaccessible’
• [x] Good story around fault injection for testing
• [x] FakeFilesystem should make it easier to diagnose resource leaks
• [x] Fix bug where FakeFilesystem.appendingSink() causes the readable file to appear empty
• [x] Fix bug where calling FakeFilesystem.createDirectories() crashes creating roots
• [x] Path.isRoot
• [x] Make Path comparable
• [x] Make Filesystem.list() return paths in sorted order?
• [x] Require FakeFilesystem.workingDirectory is an absolute path
• [x] Test case for API as used from Java
• [x] Figure out how to publish when no platforms support both macOS and Windows compilation
• [x] Move FakeFilesystem to okio.testing
• [x] Decide whether to rename Filesystem to FileSystem
• [x] Move JVM-only functions on Paths to Path, including expect/actual work
• [x] Final name for windowsLimitations parameter on FakeFilesystem
• [x] Decide whether to rename Path’s slash parameter to an enum or boolean like windowsy.
• [x] Decide on names of write and read functions

I know #260 was closed, but I don't really understand the issue, given that the initialization of the cipher is outside of the responsibility of the stream implementation. Let me know if I this is a fool's errand, but I figure this was at least worth a shot. This would be very useful for applications which treat large amounts of ciphered data, because the usage of internal segments would be a big improvement for memory usage, not to mention the simple fact of having cryptography properly included in the okio ecosystem for piping.

Closes #490

Created a sample Throttler which can throttle Sources and Sinks to a desired throughput. Multiple Sources and Sinks can be attached to the same Throttler and their combined throughput will not exceed the desired throughput. Multiple Throttlers can also be used on the same Source or Sink and they will all be honored.

Added some basic unit tests to verify desired behavior.

Would be nice for readability if we could use Kotlin extension properties for not having to nest the Okio.source(...), Okio.sink(...), Okio.buffer(...) methods.
That way one could intuitively navigate to a File's BufferedSink using file.sink.buffered.

@swankjesse's Okio file copy example on Twitter would then be reduced to this:

from.source.use { source ->
    to.sink.buffered.use { sink ->
        sink.writeAll(source)
    }
}

Extension properties

val Source.buffered: BufferedSource
    get() = Okio.buffer(this)
val Sink.buffered: BufferedSink 
    get() = Okio.buffer(this)
val OutputStream.sink: Sink
    get() = Okio.sink(this)
val Socket.sink: Sink 
    get() = Okio.sink(this)
val File.sink: Sink
    get() = Okio.sink(this)
val Path.sink: Sink
    get() = Okio.sink(this)
val File.appendingSink: Sink 
    get() = Okio.appendingSink(this)
val InputStream.source: Source 
    get() = Okio.source(this)
val Socket.source: Source 
    get() = Okio.source(this)
val File.source: Source
    get() = Okio.source(this)
val Path.source: Source
    get() = Okio.source(this)
val BufferedSink.outputStream: OutputStream 
    get() = outputStream()
val BufferedSource.inputStream: InputStream
    get() = inputStream()

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | androidx.test:runner | 1.4.0 -> 1.5.1 | | | | |

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

I've been observing failures like this:

expected 2022-12-31T16:00:40.999Z in 2022-12-31T16:00:41Z..2022-12-31T16:00:43Z (relaxed from 2022-12-31T16:00:41.002314Z..2022-12-31T16:00:41.002365Z)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | org.jetbrains.dokka:dokka-gradle-plugin | 1.7.10 -> 1.7.20 | | | | |

Release Notes

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | androidx.test.ext:junit | 1.1.3 -> 1.1.4 | | | | |

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

Towards https://github.com/square/okio/issues/1169

For discussion

General approach from

• https://paluch.biz/blog/183-carrier-kernel-thread-pinning-of-virtual-threads-project-loom.html
• https://github.com/square/okhttp/pull/7367

AsyncTimout's Watchdog's method run has a synchronized block inside a try catch.

https://github.com/square/okio/blob/master/okio/src/jvmMain/kotlin/okio/AsyncTimeout.kt

This is causing apps to crash when using Android Gradle Plugin 7.2.0 and above and testCoverageEnabled flag enabled.

taking out the synchronized of the try catch should solve the issue?