We are using ktunnel to use xdebug against our remote php pods. It works great, but we noticed that all connections are left open indefinitely.

The ktunnel logs show errors that seem to be related:

INFO[2020-07-22 22:37:08 closing session 8b033568-e519-46ea-a42d-164b12a3c46d; err: EOF 
ERRO[2020-07-22 22:37:14 8b033568-e519-46ea-a42d-164b12a3c46d; session not found in openRequests 
ERRO[2020-07-22 22:37:15 8b033568-e519-46ea-a42d-164b12a3c46d; session not found in openRequests 
INFO[2020-07-22 22:38:02 closing session 407cef1f-3cf7-4505-83b6-4f05f665388f; err: EOF 
ERRO[2020-07-22 22:38:11 407cef1f-3cf7-4505-83b6-4f05f665388f; session not found in openRequests 
ERRO[2020-07-22 22:38:12 407cef1f-3cf7-4505-83b6-4f05f665388f; session not found in openRequests 

How do we resolve this?

Hey,

I've got it set up and it works, but the request/response round-trip is intermittently very, very slow, or hangs and doesn't complete.

I have no idea what kinds of logs/tests to try to give more meaningful information about this issue, if there are any suggestions I'd be happy to undertake them.

Cheers!

Trying to use ktunnel with Rancher deployed pod (the example pod ffrom this repo). The following error is displayed: INFO[0001] Injecting tunnel sidecar to dparsons/pyremotedebug INFO[0002] Waiting for deployment to be ready INFO[0007] Waiting for port forward to finish ERRO[0007] error upgrading connection: error creating request: parse https://rancher.plaidcloud.io%2Fk8s%2Fclusters%2Fc-wfdqx/api/v1/namespaces/dparsons/pods/pyremotedebug-9ff9fc4c-trbcg/portforward: invalid URL escape "%2F"

Hi @omrikiei, it's me again 🙈

I am currently trying to forward a simple local HTTP server into the cluster. Service, deployment and kTunnel (28688 -> 28688) are always established correctly and do not show any errors. Nevertheless the ktunnel server very often does not forward any requests to the local ktunnel client. Requests end in a timeout, although no errors can be seen.

Steps to reproduce

1. download latest (brew) binary 1.1.10 for macos
2. execute ktunnel -v expose myapp 8080:8080
3. see:

INFO[0000] Exposed service's cluster ip is: 10.99.192.110 
INFO[0000] waiting for deployment to be ready           
INFO[0003] Waiting for port forward to finish           
INFO[0004] Forwarding from 127.0.0.1:28688 -> 28688
Forwarding from [::1]:28688 -> 28688 
INFO[0004] starting tcp tunnel from source 8080 to target 8080 

4. Perform a http request to service inside kubernetes (other) pod via curl -v http://myapp.default.svc.cluster.local:8080/ (ip gets resolved to exposed service cluster ip) no response, no incoming request to local server, no session created)
5. CTRL+C on client side

INFO[0027] Got exit signal, closing client tunnels and removing k8s objects 
INFO[0027] Deleting service travellerapp                
WARN[0027] error reading from stream: %vrpc error: code = Canceled desc = grpc: the client connection is closing 
INFO[0027] Deleting deployment myapp  

kubectl version

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T14:25:20Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.4", GitCommit:"67d2fcf276fcd9cf743ad4be9a9ef5828adc082f", GitTreeState:"clean", BuildDate:"2019-09-18T14:41:55Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}

In a few cases the server forwards the request to the client and the HTTP request is successfully answered. After that the CPU load of the (local) ktunnel client increases to 500-600% (6 cores). The client still responds (but no further tunnel requests are forwarded), but the CPU usage does not decrease.

The cpu usage increases after the client logs

INFO[0005] b638fa36-d3ad-4cfd-8881-38b5cf9a8cca; new session; connecting to port 8080 

Please let me know how I can help you to reproduce the bug. Thank you! Markus

i was trying ktunnel to inject deployment but not working

command used:-ktunnel inject deployment flask-deployment 80

Error

"status": "Failure",
  "message": "pods \"flask-deployment-6c86b6f988-vkfh8\" is forbidden: User \"system:anonymous\" cannot get resource \"pods/portforward\" in API group \"\" in the namespace \"default\"",
  "reason": "Forbidden",
  "details": {
    "name": "flask-deployment-6c86b6f988-vkfh8",
    "kind": "pods"
  },

I tried to use the expose command to forward traffic from a kubernetes service to a local port:

ktunnel expose myservice 80:8000

Access from another pod in the same cluster using curl just says:

Failed to connect to myservice port 80: Connection refused

It looks like the service will be created with a source port of 80 and a target port of 8000, but the pod is listening on port 80. When I change the service manually and set source and target port to 80, it works.

When we try to use ktunnel to establish a connection for using xdebug in one of our containers, it works for about 15 seconds before failing with this error:

INFO[2020-04-09 16:32:45] f73e9c2d-ffdb-48dd-aa74-a4f8be552086; new session; connecting to port 9000
ERRO[2020-04-09 16:32:58] f73e9c2d-ffdb-48dd-aa74-a4f8be552086; failed reading from socket, exiting: read tcp 127.0.0.1:55793->127.0.0.1:9000: read: connection reset by peer

The ktunnel command we were using was ktunnel inject -n poor-cannon deployment api 8999:8999

ktunnel version: 1.2.2

Hi @omrikiei :)

Its me again, this time with the following:

2020-02-06 09:52:13 >| $ ktunnel expose myWebapp 8084:8084 -n myNamespace
INFO[0000] Exposed service's cluster ip is: 10.31.10.100
INFO[0000] waiting for deployment to be ready           
INFO[0007] All pods located for port-forwarding         
INFO[0007] Waiting for port forward to finish           
INFO[0007] Forwarding from 127.0.0.1:28688 -> 28688
Forwarding from [::1]:28688 -> 28688 
INFO[0007] starting tcp tunnel from source 8084 to target 8084 
E0206 09:52:21.785327   10875 portforward.go:400] an error occurred forwarding 28688 -> 28688: error forwarding port 28688 to pod 3ddcdef600c6c0d884e3d71f5600a350623786083794ac4ef3a330c5ddc2944b, uid : exit status 1: 2020/02/06 07:52:21 socat[165522] E connect(5, AF=2 127.0.0.1:28688, 16): Connection refused
ERRO[0007] Error sending init tunnel request: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection closed 
INFO[0023] Got exit signal, closing client tunnels and removing k8s objects 
INFO[0023] Deleting service myWebapp

I'll appreciate :heart_eyes: any clues as always :)

Kind regards, Boris

Image is v1.3.6

Kubernetes side, deployment/pod seems to start up just fine:

k get pods
NAME                       READY   STATUS    RESTARTS   AGE
ktunnel-77b65858f8-w9zg5   1/1     Running   0          35s

k get deployments
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
ktunnel   1/1     1            1           37s

Edit: just tried with v1.3.3 also since I remember that working for sure last time I used this (It's me from #36 :D) issue persists just the same... I've checked I haven't got any lingering ktunnel resources server-side as well

Edit: k logs on the server pod shows:

INFO[2021-04-12 15:35:20.451] Starting to listen on port 28688          

I'm trying to proxy to a local web server, but it looks like ktunnel doesn't behave well with large files. Here's some output from curl:

❯ curl https://proxy.mbuffett.com/_next/static/chunks/main.js | wc -c
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  8 4295k    8  377k    0     0  73862      0  0:00:59  0:00:05  0:00:54 73862
curl: (18) transfer closed with 4012193 bytes remaining to read
386672

❯ curl localhost:8085/_next/static/chunks/main.js | wc -c            
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4295k  100 4295k    0     0   209M      0 --:--:-- --:--:-- --:--:--  209M
4398865

Please, no judgement for the 4MB JS file :smile: I do see these logs from ktunnel about buffer sizes:

INFO[2020-07-30 07:59:34] increasing buffer size to 65536              
INFO[2020-07-30 07:59:34] increasing buffer size to 131072

Wonder if there's anything I can do about this limitation? Thanks!

And awesome program, exactly what I was looking for and easy to set up

failed connecting to localhost on port 9002 scheme tcp

ktunnel version 1.2.4 (1.2.5 actually)

my PHPStorm is listening port 9002.

➜  ~ lsof -i :9002
COMMAND    PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
phpstorm 50781 cyao  465u  IPv6 0x32e9aa96a4c0d673      0t0  TCP *:pichat (LISTEN)

And I also tried to start a TCP server to respond to the connection.

➜  ~ nc -vl 9002

same error.

➜  ~ ktunnel -v inject deployment my-php-app 9000:9002 -n playground
INFO[2020-05-01 00:59:00] Injecting tunnel sidecar to playground/my-php-app
INFO[2020-05-01 00:59:01] Waiting for deployment to be ready
INFO[2020-05-01 00:59:23] All pods located for port-forwarding
DEBU[2020-05-01 00:59:23] Injecting to this pods: [my-php-app-6995b778b-8fzm5]
INFO[2020-05-01 00:59:23] Waiting for port forward to finish
INFO[2020-05-01 00:59:24] Forwarding from 127.0.0.1:28688 -> 28688
Forwarding from [::1]:28688 -> 28688
INFO[2020-05-01 00:59:24] starting tcp tunnel from source 9000 to target 9002
DEBU[2020-05-01 00:59:25] attempting to receive from stream
DEBU[2020-05-01 00:59:33] 780fece3-8339-40f4-bcf4-2da5a60fd5de; got session from server: 499<?xml version="1.0" encoding="iso-8859-1"?>
<init xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" fileuri="file:///var/www/html/www/public/index.php" language="PHP" xdebug:language_version="7.3.17" protocol_version="1.0" appid="38" idekey="PHPSTORM"><engine version="2.7.2"><![CDATA[Xdebug]]></engine><author><![CDATA[Derick Rethans]]></author><url><![CDATA[https://xdebug.org]]></url><copyright><![CDATA[Copyright (c) 2002-2019 by Derick Rethans]]></copyright></init>
INFO[2020-05-01 00:59:33] 780fece3-8339-40f4-bcf4-2da5a60fd5de; new session; connecting to port 9002
ERRO[2020-05-01 00:59:33] failed connecting to localhost on port 9002 scheme tcp

I have a fairly simple use-case but struggling to work out the series of commands to make this happen. In a dev environment with several namespaces each serving a php web app declared using a statefulset configuration, I want to reverse forward port 9003 to my local xdebug client. I expect

ktunnel inject deployment <name_of_statefulset> 9003 -v
INFO[0000] Injecting tunnel sidecar to default/owl-app  
FATA[0001] failed injecting sidecar: deployments.apps "owl-app" not found 

to use the existing statefulset or for ktunnel to provide an ktunnel inject statefulset command.

If there is a workaround or a supported way to accomplish this, help would be appreciated.

Thanks.

Hello,

I'm trying to use the tunnel krew plugin on Mac M1 to tunnel to a local Kube cluster (Docker for Mac)

It seems that there is not a published version for arm64 on quay.io, and if I try to use the amd64 through rosetta, it segfaults and does not create the tunnel:

Output from running the docker container directly:

docker run --rm quay.io/omrikiei/ktunnel:v1.4.8
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

Output from running the Krew plugin:

kubectl tunnel expose --namespace=default --tls xdebug-svc 9003
INFO[0000] Exposed service's cluster ip is: 10.99.222.125
.INFO[0000] waiting for deployment to be ready
......
INFO[0002] port forwarding to https://kubernetes.docker.internal:6443/api/v1/namespaces/default/pods/xdebug-svc-678c857f7c-77n9n/portforward
INFO[0002] Waiting for port forward to finish
INFO[0002] Forwarding from 127.0.0.1:28688 -> 28688
Forwarding from [::1]:28688 -> 28688
INFO[2022-09-22 18:03:02.586] starting tcp tunnel from source 9003 to target 9003
E0922 18:03:02.617812   79680 portforward.go:400] an error occurred forwarding 28688 -> 28688: error forwarding port 28688 to pod e187a3212e5042c7226cf8ed49fb5b2585673d4701e6847e22aa1d2ae63d6539, uid : exit status 1: 2022/09/22 16:03:02 socat[19739] E connect(16, AF=2 127.0.0.1:28688, 16): Connection refused
ERRO[2022-09-22 18:03:02.617] Error sending init tunnel request: rpc error: code = Unavailable desc = connection closed
E0922 18:03:03.630357   79680 portforward.go:400] an error occurred forwarding 28688 -> 28688: error forwarding port 28688 to pod e187a3212e5042c7226cf8ed49fb5b2585673d4701e6847e22aa1d2ae63d6539, uid : exit status 1: 2022/09/22 16:03:03 socat[19740] E connect(16, AF=2 127.0.0.1:28688, 16): Connection refused
^CINFO[0004] Got exit signal, closing client tunnels and removing k8s objects
INFO[0004] Deleting service xdebug-svc
INFO[0004] Deleting deployment xdebug-svc

Logs from the pod:

│ qemu: uncaught target signal 11 (Segmentation fault) - core dumped                                                                                                     

Would it be possible to publish the arm64 image?

Thanks for the awesome work on this tool!

When using ktunnel with OpenShift, for me ktunnel expose does not reliably work because the Deployment created has RunAsUser, which OCP does not allow unless you adjust it's security configuration.

OpenShift runs containers with non-root user by default, and the Deployment works fine when I manually remove the RunAsUser.

So far the bug, I am open to providing a PR, but maybe you already have a solution in mind?

It is mentioned in the README "Although ktunnel is identified with kubernetes, it can also be used as a reverse tunnel on any other remote system"

Is there anyway I can use ktunnel to expose local resources(webserver) so that they can be accessed from within the GitHub codespaces container?

Thanks