资源说明:Runs something "Over there"
Apart from these common connection options, some protocols define additional protocol-specific connection options. These are documented below, with the corresponding protocol.
## LOCAL
The local protocol implementation uses the local file manipulation and local process execution capabilities built-in to Java. The [__os__](#os) connection option is hardcoded to the operating system of the local host and the [__tmp__](#tmp) connection option defaults to the system temporary directory as specified by the `java.io.tmpdir` system property. There are no protocol-specific connection options.
## SSH
The SSH protocol implementation of Overthere uses the [SSH](http://en.wikipedia.org/wiki/Secure_Shell) protocol to connect to remote hosts to manipulate files and execute commands. Most Unix systems already have an SSH server installed and configured and a number of different SSH implementations are available for Windows although not all of them are supported by Overthere.
To connect to a remote host using the SSH protocol, you will need to install an SSH server on that remote host. For Unix and Windows platforms, we recommend [OpenSSH](http://www.openssh.com/). It is included in all Linux distributions and most other Unix flavours. For Windows platforms three SSH servers are supported:
* OpenSSH on [Cygwin](http://www.cygwin.com/). We recommend [COPSSH](http://www.itefix.no/i2/copssh) as a convenient packaging of OpenSSH and Cygwin. It is a free source download but since 22/11/2011 the binary installers are a paid solution.
* [WinSSHD](http://www.bitvise.com/winsshd) is a commercial SSH server that has a lot of configuration options.
* Install microsoft [OpenSSHD](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse) on your Windows host and make sure the OpenSSH Server service is enabled.
__N.B.:__ The __SFTP__, __SCP__, __SU__, __SUDO__ and __INTERACTIVE_SUDO__ connection types are only available for Unix hosts. To use SSH with z/OS hosts, use the __SFTP__ connection type. To use SSH with Windows hosts, choose either the __SFTP_CYGWIN__ or the __SFTP_WINSSHD__ or the __SFTP_OPENSSHD__ connection type.
### Compatibility
Overthere uses the [sshj](https://github.com/hierynomus/sshj) library for SSH and supports all algorithms and formats supported by that library:
* Ciphers: ``aes{128,192,256}-{cbc,ctr}``, ``blowfish-{cbc,ctr}``, ``3des-{cbc,ctr}``, ``twofish{128,192,256}-{cbc,ctr}``, ``twofish-cbc``, ``serpent{128,192,256}-{cbc,ctr}``, ``idea-{cbc,ctr}``, ``cast128-{cbc,ctr}``, ``arcfour``, ``arcfour{128,256}``
* Key Exchange methods: ``diffie-hellman-group{1,14}-sha1``, ``diffie-hellman-group-exchange-sha{1,256}``, ``ecdh-sha2-nistp{256,384,521}``, ``curve25519-sha256@libssh.org``
* Signature formats: ``ssh-rsa``, ``ssh-dss``, ``ecdsa-sha2-nistp256``, ``ssh-ed25519``
* MAC algorithms: ``hmac-md5``, ``hmac-md5-96``, ``hmac-sha1``, ``hmac-sha1-96``, ``hmac-sha2-256``, ``hmac-sha2-512``
* Compression algorithms: ``zlib`` and ``zlib@openssh.com`` (delayed zlib)
* Private Key file formats: ``pkcs8`` encoded (the format used by [OpenSSH](http://www.openssh.com/))
### SSH host setup
#### SFTP
To use the __SFTP__ connection type, make sure SFTP is enabled in the SSH server. This is enabled by default in most SSH servers.
#### SFTP_CYGWIN
To use the __SFTP_CYGWIN__ connection type, install [COPSSH](http://www.itefix.no/i2/copssh) on your Windows host. In the COPSSH control panel, add the users as which you want to connect and select _Linux shell and Sftp_ in the _shell_ dropdown box. Check _Password authentication_ and/or _Public key authentication_ depending on the authentication method you want to use.
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to Cygwin-style paths, e.g. `/cygdrive/C/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SFTP_WINSSHD To use the __SFTP_WINSSHD__ connection type, install [WinSSHD](http://www.bitvise.com/winsshd) on your Windows host. In the Easy WinSSHD Settings control panel, add the users as which you want to connect, check the _Login allowed_ checkbox and select _Allow full access_ in the _Virtual filesystem layout_ dropdown box. Alternatively you can check the _Allow login to any Windows account_ to allow access to all Windows accounts.
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to WinSSHD-style paths, e.g. `/C/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SFTP_OpenSSHD To use the __SFTP_OpenSSHD__ connection type, install [OpenSSHD](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse) on your Windows host and make sure the OpenSSH Server service is enabled.
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to OpenSSHD-style paths, e.g. `/C:/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SUDO and INTERACTIVE_SUDO To use the __SUDO__ connection type, the `/etc/sudoers` configuration will have to be set up in such a way that the user configured with the connection option __username__ can execute the commands below as the user configured with the connection option __sudoUsername__. The arguments passed to these commands depend on the exact usage of the Overthere connection. Check the `INFO` messages on the `com.xebialabs.overthere.ssh.SshConnection` category to see what commands get executed. * `chmod` * `cp` * `ls` * `mkdir` * `mv` * `rm` * `rmdir` * `tar` * Any other command that you want to execute. The commands mentioned above must be configured with the __NOPASSWD__ setting in the `/etc/sudoers` file. Otherwise you will have to use the __INTERACTIVE_SUDO__ connection type. When the __INTERACTIVE_SUDO__ connection type is used, every line of the output will be matched against the regular expression configured with the __sudoPasswordPromptRegex__ connection option. If a match is found, the value of the __password__ connection option is sent. ### Troubleshooting SSH connections This section lists a number of common configuration errors that can occur when using Overthere with SSH. If you run into other connectivity issues when using Overthere, pease let us know by [creating a ticket](https://github.com/xebialabs/overthere/issues) or by [sending us a pull request](https://github.com/xebialabs/overthere/pulls). #### Cannot start a process on an SSH server because the server disconnects immediately. If the terminal type requested using the [__allocatePty__](#ssh_allocatePty) connection option or the [__allocateDefaultPty__](#ssh_allocateDefaultPty) connection option is not recognized by the SSH server, the connection will be dropped. Specifically, the `dummy` terminal type configured by [__allocateDefaultPty__] connection option, will cause OpenSSH on AIX and WinSSHD to drop the connection. Try a safe terminal type such as `vt220` instead. To verify the behaviour of your SSH server with respect to pty allocation, you can manually execute the
## SMB 2.x/CIFS, WinRM and Telnet
### SMB 2.x
The SMB 2.x protocol implementation of Overthere uses the prevalent [SMB protocol](http://en.wikipedia.org/wiki/Server_Message_Block) for file manipulation and, depending on the settings, using either [WinRM](http://en.wikipedia.org/wiki/WS-Management) or [Telnet](http://en.wikipedia.org/wiki/Telnet) for process execution.
### CIFS
The CIFS protocol implementation of Overthere uses public variant of the original Server Message Block (SMB) protocol developed by Microsoft known as [CIFS protocol](http://en.wikipedia.org/wiki/Server_Message_Block), for file manipulation and, depending on the settings, uses either [WinRM](http://en.wikipedia.org/wiki/WS-Management) or [Telnet](http://en.wikipedia.org/wiki/Telnet) for process execution. CIFS is widely regarded as an obsolete protocol and users are encouraged to prefer SMB protocol over CIFS. Support for CIFS is deprecated and will be removed from subsequent releases.
### SMB 2.x and CIFS
These protocols are only supported for Windows hosts, you will most likely not need to install new software although you might need to enable and configure some services:
* The built-in file sharing capabilities of Windows are based on CIFS/SMB and are therefore available and enabled by default.
* WinRM is available on Windows Server 2008 and up. Overthere supports basic authentication for local accounts and Kerberos authentication for domain accounts. Overthere has a built-in WinRM library that can be used from all operating systems by setting the [**connectionType**](#smb_cifs_connectionType) connection option to __WINRM_INTERNAL__. When connecting from a host that runs Windows, or when using a "winrs proxy host" that runs Windows, the native WinRM capabilities of Windows, i.e. the `winrs` command, can be used by setting the [**connectionType**](#smb_cifs_connectionType) connection option to __WINRM_NATIVE__.
* A Telnet Server is available on all Windows Server versions although it might not be enabled.
To connect to a remote host using the __SMB__ or __CIFS__ protocol, ensure the host is reachable on port 445.
If you will be connecting as an administrative user, ensure the administrative shares are configured. Otherwise, ensure that the user you will be using to connect has access to shares that correspond to the directory you want to access and that the [__pathShareMappings__](#smb_cifs_pathShareMappings) connection option is configured accordingly.
### Password limitations
Due to a limitation of the `winrs` command, passwords containing a single quote (`'`) or a double quote (`"`) cannot be used when using the __WINRM_NATIVE__ connection type.
### Domain accounts
Windows domain accounts are supported by the __WINRM_INTERNAL__, __WINRM_NATIVE__ and __TELNET__ connection types, but the syntax of the username is different:
* For the __WINRM_INTERNAL__ connection type, domain accounts must be specified using the new-style domain syntax, e.g. `USER@FULL.DOMAIN`.
* For the __TELNET__ connection type, domain accounts must be specified using the old-style domain syntax, e.g `DOMAIN\USER`.
* For the __WINRM_NATIVE__ connection type, domain accounts may be specified using either the new-style (`USER@FULL.DOMAIN`) or old-style (`DOMAIN\USER`) domain syntax.
* For all three connection types, local accounts must be specified without an at-sign (`@`) or a backslash (`\`).
__N.B.:__ When using domain accounts with the __WINRM_INTERNAL__ connection type, the Kerberos subsystem of the Java Virtual Machine must be configured correctly. Please read the section on how to set up Kerberos [for the source host](#smb_cifs_host_setup_krb5) and [the remote hosts](#smb_cifs_host_setup_spn).
### Administrative shares
By default Overthere will access the [administrative shares](http://en.wikipedia.org/wiki/Administrative_share) on the remote host. These shares are only accessible for users that are part of the __Administrators__ on the remote host. If you want to access the remote host using a regular account, use the [__pathShareMapping__](#smb_cifs_pathShareMappings) connection option to configure the shares to use for the paths Overthere will be connecting to. Of course, the user configured with the __username__ connection option should have access to those shares and the underlying directories and files.
__N.B.:__ Overthere will take care of the translation from Windows paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to SMB URLs that use the administrative shares, e.g. `smb://username:password@hostname/C$/Program%20Files/IBM/WebSphere/AppServer` (which corresponds to the UNC path `\\hostname\C$\Program Files\IBM\WebSphere\AppServer`), so that your code can use Windows style paths.
### Host setup
#### TELNET
To use the __TELNET__ connection type, you'll need to enable and configure the Telnet Server according to these instructions:
1. (Optional) If the Telnet Server is not already installed on the remote host, add it using the __Add Features Wizard__ in the __Server Manager__ console.
1. (Optional) If the remote host is running Windows Server 2003 SP1 or an x64-based version of Windows Server 2003, install the Telnet server according to [these instructions from the Microsoft Support site](http://support.microsoft.com/kb/899260).
1. Enable the Telnet Server Service on the remote host according to these instructions on the Microsoft Technet site.
1. After you have started the Telnet Server, open a command prompt as the __Administrator__ user on the remote host and enter the command `tlntadmn config mode=stream` to enable stream mode.
When the Telnet server is enabled any user that is in the __Administrators__ group or that is in the __TelnetClients__ group and that has the __Allow logon locally__ privilege can log in using Telnet. See the Microsoft Technet to learn how to grant a user or group the right to logon locally on Windows Server 2008 R2.
#### WINRM (WINRM_INTERNAL and WINRM_NATIVE)
_For a PowerShell script to do what is described below in one go, check [Richard Downer's blog](http://www.frontiertown.co.uk/2011/12/overthere-control-windows-from-java/)_
To use the __WINRM_INTERNAL__ or the __WINRM_NATIVE__ connection type, you'll need to setup WinRM on the remote host by following these instructions:
1. If the remote host is running Windows Server 2003 SP1 or SP2, or Windows XP SP2, install the [WS-Management v.1.1 package](http://support.microsoft.com/default.aspx?scid=kb;EN-US;936059&wa=wsignin1.0).
1. If the remote host is running Windows Server 2003 R2, go to the __Add/Remove System Components__ feature in the __Control Panel__ and add WinRM under the section __Management and Monitoring Tools__. Afterwards install the [WS-Management v.1.1 package](http://support.microsoft.com/default.aspx?scid=kb;EN-US;936059&wa=wsignin1.0) to upgrade the WinRM installation.
1. If the remote host is running Windows Vista or Windows 7, the __Windows Remote Management (WS-Management)__ service is not started by default. Start the service and change its Startup type to __Automatic (Delayed Start)__ before proceeding with the next steps.
1. On the remote host, open a Command Prompt (not a PowerShell prompt!) using the __Run as Administrator__ option and paste in the following lines when using the __WINRM_INTERNAL__ connection type:
winrm quickconfig
y
winrm set winrm/config/service/Auth @{Basic="true"}
winrm set winrm/config/service @{AllowUnencrypted="true"}
winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"}
Or the following lines when using the __WINRM_NATIVE__ connection type:
winrm quickconfig
y
winrm set winrm/config/service/Auth @{Basic="true"}
winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"}
Or keep reading for more detailed instructions. :-)
1. Run the quick config of WinRM to start the Windows Remote Management service, configure an HTTP listener and create exceptions in the Windows Firewall for the Windows Remote Mangement service:
winrm quickconfig
__N.B.:__ The Windows Firewall needs to be running to run this command. See [Microsoft Knowledge Base article #2004640](http://support.microsoft.com/kb/2004640).
1. (Optional) By default basic authentication is disabled in WinRM. Enable it if you are going to use local accounts to access the remote host:
winrm set winrm/config/service/Auth @{Basic="true"}
1. (Optional) By default Kerberos authentication is enabled in WinRM. Disable it if you are __not__ going to use domain accounts to access the remote host:
winrm set winrm/config/service/Auth @{Kerberos="false"}
__N.B.:__ Do not disable Negotiate authentication as the `winrm` command itself uses that to configure the WinRM subsystem!
1. (Only required for __WINRM_INTERNAL__ or when the connection option [**winrsUnencrypted**](#smb_cifs_winrsUnencrypted) is set to `true`) Configure WinRM to allow unencrypted SOAP messages:
winrm set winrm/config/service @{AllowUnencrypted="true"}
1. Configure WinRM to provide enough memory to the commands that you are going to run, e.g. 1024 MB:
winrm set winrm/config/winrs @{MaxMemoryPerShellMB="1024"}
__N.B.:__ This is not supported by WinRM 3.0, included with the Windows Management Framework 3.0. This update [has been temporarily removed from Windows Update](http://blogs.msdn.com/b/powershell/archive/2012/12/20/windows-management-framework-3-0-compatibility-update.aspx) because of numerous incompatiblity issues with other Microsoft products. However, if you have already installed WMF 3.0 and cannot downgrade, [Microsoft Knowledge Base article #2842230](http://support.microsoft.com/kb/2842230) describes a hotfix that can be installed to re-enable the `MaxMemoryPerShellMB` setting.
1. To use the __WINRM_INTERNAL__ or __WINRM_NATIVE__ connection type with HTTPS, i.e. [__winrmEnableHttps__](#smb_cifs_winrmEnableHttps) set to `true`, follow the steps below:
1. (Optional) Create a self signed certificate for the remote host by installing `selfssl.exe` from [the IIS 6 resource kit](http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=17275) and running the command below or by following the instructions [in this blog by Hans Olav](http://www.hansolav.net/blog/SelfsignedSSLCertificatesOnIIS7AndCommonNames.aspx):
C:\Program Files\IIS Resources\SelfSSL>selfssl.exe /T /N:cn=HOSTNAME /V:3650
Microsoft (R) SelfSSL Version 1.0
Copyright (C) 2003 Microsoft Corporation. All rights reserved.
Do you want to replace the SSL settings for site 1 (Y/N)?Y
The self signed certificate was successfully assigned to site 1.
1. Open a PowerShell window and enter the command below to find the thumbprint for the certificate for the remote host:
PS C:\Windows\system32> Get-childItem cert:\LocalMachine\Root\ | Select-String -pattern HOSTNAME
[Subject]
CN=HOSTNAME
[Issuer]
CN=HOSTNAME
[Serial Number]
527E7AF9142D96AD49A10469A264E766
[Not Before]
5/23/2011 10:23:33 AM
[Not After]
5/20/2021 10:23:33 AM
[Thumbprint]
5C36B638BC31F505EF7F693D9A60C01551DD486F
1. Create an HTTPS WinRM listener for the remote host with the thumbprint you've just found:
winrm create winrm/config/Listener?Address=*+Transport=HTTPS @{Hostname="HOSTNAME"; CertificateThumbprint="THUMBPRINT"}
For more information on WinRM, please refer to the online documentation at Microsoft's DevCenter. As a quick reference, have a look at the list of useful commands below:
* Do a quickconfig for WinRM with HTTPS: `winrm quickconfig -transport:https`
* View the complete WinRM configuration: `winrm get winrm/config`
* View the listeners that have been configured: `winrm enumerate winrm/config/listener`
* Create an HTTP listener: `winrm create winrm/config/listener?Address=*+Transport=HTTP` (also done by `winrm quickconfig`)
* Allow all hosts to connect to the WinRM listener: `winrm set winrm/config/client @{TrustedHosts="*"}`
* Allow a fixed set of hosts to connect to the WinRM listener: `winrm set winrm/config/client @{TrustedHosts="host1,host2..."}`
#### Kerberos - source host
__N.B.:__ You will only need to configure Kerberos if you are going to use Windows domain accounts to access the remote host with the __WINRM_INTERNAL__ connection type.
In addition to the setup described in [the WINRM section](#smb_cifs_host_setup_winrm), using Kerberos authentication requires that you follow the [Kerberos Requirements for Java](http://docs.oracle.com/javase/6/docs/technotes/guides/security/jgss/tutorials/KerberosReq.html) on the host from which the Overthere connections are initiated, i.e. the source host.
Create a file called `krb5.conf` (Unix) or `krb5.ini` (Windows) with at least the following content:
[realms]
EXAMPLE.COM = {
kdc = KDC.EXAMPLE.COM
}
Replace the values with the name of your domain/realm and the hostname of your domain controller (multiple entries can be added to allow the Overthere source host to connect to multiple domains) and place the file in the default location for your operating system:
* Linux: `/etc/krb5.conf`
* Solaris: `/etc/krb5/krb5.conf`
* Windows: `C:\Windows\krb5.ini`
Alternatively, place the file somewhere else and add the following Java system property to the command line: `-Djava.security.krb5.conf=/path/to/krb5.conf`. Replace the path with the location of the file you just created.
See [the Kerberos V5 System Administrator's Guide at MIT](http://web.mit.edu/kerberos/krb5-1.10/krb5-1.10.6/doc/krb5-admin.html#krb5_002econf) for more information on the `krb5.conf` format.
#### Kerberos - remote host
__N.B.:__ You will only need to configure Kerberos if you are going to use Windows domain accounts to access the remote host with the __WINRM_INTERNAL__ connection type.
By default, Overthere 2.1.0 and up will request access to a Kerberos service principal name of the form
# Table of Contents * [Introduction](#introduction) * [Getting Overthere](#getting-overthere) * [Depending on Overthere](#depending-on-overthere) * [Building Overthere](#building-overthere) * [Running the Examples](#running-the-examples) * [Programming Overthere](#programming-overthere) * [Configuring Overthere](#configuring-overthere) * [Protocols](#protocols) * [Common connection options](#common-connection-options) * [Local](#local) * [SSH](#ssh) * [Host setup](#ssh-host-setup) * [Troubleshooting](#troubleshooting-ssh-connections) * [Connection options](#ssh-connection-options) * [SMB 2.x/CIFS, WinRM and Telnet](#smb-2x-and-cifs) * [SMB 2.x](#smb-2x) * [CIFS](#cifs) * [Host setup](#smb_cifs_host_setup) * [Troubleshooting](#smb_cifs_troubleshooting) * [Connection options](#smb_cifs_connection_options) * [Jumpstations: SSH tunnels and HTTP proxies](#jumpstations-ssh-tunnels-and-http-proxies) * [SSH connection to GCP instances](#ssh-connection-to-gcp-instances) * [GCP OsLogin: Using GCP's service account with OsLogin](#gcp-oslogin-using-gcps-service-account-with-oslogin) * [GCP Metadata: Using GCP's service account with metadata](#gcp-metadata-using-gcps-service-account-with-metadata) * [GCP Generic SSH options](#gcp-generic-ssh-options) * [GCP Credentials](#gcp-credentials) * [Release History](#release-history) # Introduction Overthere is a Java library to manipulate files and execute processes on remote hosts, i.e. do stuff "over there". Overthere was originally developed for and is used in the [XebiaLabs](http://xebialabs.com/) deployment automation product Deployit as a way to perform tasks on remote hosts, e.g. copy configuration files, install EAR files or restart web servers. Another way of looking at it is to say that Overthere gives you `java.io.File` and `java.lang.Process` as they should've been: as interfaces, created by a factory and extensible through an SPI mechanism. Overthere is available under the [GPLv2 with XebiaLabs FLOSS License Exception](https://raw.github.com/xebialabs/overthere/master/LICENSE). __P.S.:__ Check the [Overthere Ohloh page](http://www.ohloh.net/p/overthere) for some interesting code analysis statistics. If you use Overthere, don't forget to tell Ohloh! And while you're at it, you might want to vote for Overthere on the [Overthere Freecode page](http://freecode.com/projects/overthere) too! ;-) # Getting Overthere To get Overthere, you have two options: 1. Add a dependency to Overthere to your project. 1. Build Overthere yourself. And, if you want, you can also run the Overthere examples used in the Overthere presentation mentioned above. Binary releases of Overthere are not provided here, but you can download it [straight from the Maven Central repository](http://search.maven.org/#artifactdetails%7Ccom.xebialabs.overthere%7Coverthere%7C5.0.8%7Cjar) if you want to. ## Depending on Overthere 1. If your project is built with Maven, add the following dependency to the pom.xml:1. If your project is built using another build tool that uses the Maven Central repository, translate these dependencies into the format used by your build tool. ## Building Overthere 1. Clone the Overthere repository. 1. On unix run the command `./gradlew clean build`, on windows run `gradlew clean build` ## Running the examples 1. Install [Maven 2.2.1](http://maven.apache.org/) or up. 1. Clone the Overthere repository. 1. Go into the `examples` directory and run the command `mvn eclipse:eclipse`. 1. Import the `examples` project into Eclipse. 1. Change the login details in the example classes (address, username and password) and run them! # Programming Overthere To program Overthere, browse the source code, check the examples and browse the Overthere Javadoc. For a more thorough introduction to Overthere, check the [presentation on Overthere](http://www.slideshare.net/vpartington/presentation-about-overthere-for-jfall-2011) that I gave for J-Fall 2011, a Java conference in the Netherlands (in English). # Configuring Overthere The protocols that Overthere uses to connect to remote hosts, such as SSH, CIFS, Telnet and WinRM, are existing protocols for which support is built into many platforms. As such you will not need to install any custom software on the remote hosts. Nevertheless in some cases the remote hosts have to be configured to correctly work with Overthere. Also, Overthere has a number of configuration features that allow you tweak the way it interfaces with the remote hosts. ## Protocols Overthere supports a number of protocols to connect to remote hosts: * [__local__](#local) - a connection to the local host. This is a wrapper around java.io.File and java.lang.Process. * [__ssh__](#ssh) - a connection using the [SSH protocol](http://en.wikipedia.org/wiki/Secure_Shell), to a Unix host, to a z/OS host, or to a Windows host running either OpenSSH on Cygwin (i.e. COPSSH) or WinSSHD or microsoft OpenSSH. * [__smb__](#smb) - a connection using the prevalent [SMB protocol](http://en.wikipedia.org/wiki/Server_Message_Block) for file manipulation and, depending on the settings, using either [WinRM](http://en.wikipedia.org/wiki/WS-Management) or [Telnet](http://en.wikipedia.org/wiki/Telnet) for process execution. This protocol is only supported for Windows hosts. * [__cifs__](#cifs) - a connection using public variant of the original Server Message Block (SMB) protocol developed by Microsoft known as [CIFS protocol](http://en.wikipedia.org/wiki/Server_Message_Block), for file manipulation and, depending on the settings, using either [WinRM](http://en.wikipedia.org/wiki/WS-Management) or [Telnet](http://en.wikipedia.org/wiki/Telnet) for process execution. This protocol is only supported for Windows hosts, CIFS is widely regarded as an obsolete protocol and users are encouraged to prefer a SMB protocol over a CIFS. Support for CIFS is deprecated and will be removed from subsequent releases. * [__ssh-jumpstation__](#jumpstations) - a special protocol type that can only be used as a jumpstation protocol, which allows a connection to be created over an [SSH jumpstation](https://en.wikipedia.org/wiki/Port_forwarding#Local_port_forwarding). * [__proxy__](#jumpstations) - a special protocol type that can only be used as a jumpstation protocol, which allows a connection to be created over an [HTTP proxy](https://en.wikipedia.org/wiki/HTTP_tunnel). ## Common connection options Apart from selecting a protocol to use, you will also need to supply a number of connection options when creating a connection. Common connection options are: com.xebialabs.overthere overthere 5.3.2
protocol | This option is only used when it is present in the connection options of a tunnel (jumpstation) connection. It indicates which protocol is used to tunnel the other connection over. This property can be set to ssh or proxy . If this option is omitted in the jumpstation options, ssh is assumed. |
---|---|
os | The operating system of the remote host. This property can be set to UNIX , WINDOWS , and ZOS and is used to
determine how to encode paths and commands and to determine the default temporary directory path. This property is required for all protocols, except
for the local protocol where it is automatically determined. |
address | The address of the remote host, either an IP address or a DNS name. This property is required for all protocols, except for the local protocol. |
port | The port to use when connecting to the remote host. The interpretation and the default value for this connection option depend on the protocol that is used. |
username | The username to use when connecting to the remote host. This property is required for all protocols, except for the local protocol. |
password | The password to use. This property is required for all protocols, except for the local protocol. |
tmp | The temporary directory. For each connection, a connection temporary directory with a name like
overthere-20111128T132600-7234435.tmp is created within this temporary directory, e.g.
/tmp/overthere-20111128T132600-7234435.tmp , to store temporary files for the duration of the connection.The default value is tmp for UNIX and z/OS hosts and C:\windows\temp for Windows hosts, except for the local protocol where
the default is the value of the java.io.tmpdir system property. |
tmpFileCreationRetries | The number of times Overthere attempts to create a temporary file with a unique name. The default value is 100 . |
tmpDeleteOnDisconnect | If set to false , the connection temporary directory is not removed when the connection. The default value is true . |
connectionTimeoutMillis | The number of milliseconds Overthere waits for a connection to a remote host to be established. The default value is 120000 , i.e.
2 minutes. |
socketTimeoutMillis | The number of milliseconds Overthere will waits when no data is received on an open connection before raising exception. The default value is 0 , i.e.
infinite timeout. |
jumpstation | If set to a non-null value, this property contains the connection options used to connect to an SSH jumpstation (See Tunnelling). Recursive configuration is possible, i.e. this property is also available for the connection options of a jumpstation. |
fileCopyCommandForUnix | The command to use when copying a file on a Unix host. The string {0} is replaced with the path of the source file, the string
{1} is replaced with the path of the destination file. The default value is cp -p {0} {1} . |
directoryCopyCommandForUnix | The command to use when copying a directory on a Unix host. The string {0} is replaced with the path of the source directory, the string
{1} is replaced with the path of the destination directory. The default value is cd {1} ; tar -cf - -C {0} . | tar xpf - . If the
tar command is not available but the find command recognizes the -depth parameter with a value, the alternative
command find {0} -depth 1 -exec cp -pr {} {1} ; may be configured. |
fileCopyCommandForWindows | The command to use when copying a file on a Windows host. The string {0} is replaced with the path of the source file, the string
{1} is replaced with the path of the destination file. The default value is copy {0} {1} /y . |
directoryCopyCommandForWindows | The command to use when copying a directory on a Windows host. The string {0} is replaced with the path of the source directory, the string
{1} is replaced with the path of the destination directory. The default value is xcopy {0} {1} /i /y /s /e /h /q . |
fileCopyCommandForZos | The command to use when copying a file on a z/OS host. The string {0} is replaced with the path of the source file, the string
{1} is replaced with the path of the destination file. The default value is cp -p {0} {1} . |
directoryCopyCommandForZos | The command to use when copying a directory on a z/OS host. The string {0} is replaced with the path of the source directory, the string
{1} is replaced with the path of the destination directory. The default value is tar cC {0} . | tar xmC {1} . . If the
tar command is not available but the find command recognizes the -depth parameter with a value, the alternative
command find {0} -depth 1 -exec cp -pr {} {1} ; may be configured. |
remoteCopyBufferSize | The buffer size to use when copying files from one connection to the other. The buffer size is taken from the _source_ file's connection. The default value is 64 KB (64*1024 bytes) . Larger values potentially break copy operations. |
remoteCharacterEncoding | The character encoding used to transcode files from one connection to the other. The default value is Operating System dependent and is set to 'UTF-8' for Windows and Unix, and to 'Cp1047' (EBCDIC) for Z/OS. |
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to Cygwin-style paths, e.g. `/cygdrive/C/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SFTP_WINSSHD To use the __SFTP_WINSSHD__ connection type, install [WinSSHD](http://www.bitvise.com/winsshd) on your Windows host. In the Easy WinSSHD Settings control panel, add the users as which you want to connect, check the _Login allowed_ checkbox and select _Allow full access_ in the _Virtual filesystem layout_ dropdown box. Alternatively you can check the _Allow login to any Windows account_ to allow access to all Windows accounts.
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to WinSSHD-style paths, e.g. `/C/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SFTP_OpenSSHD To use the __SFTP_OpenSSHD__ connection type, install [OpenSSHD](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_install_firstuse) on your Windows host and make sure the OpenSSH Server service is enabled.
__N.B.:__ Overthere will take care of the translation from Windows style paths, e.g. `C:\Program Files\IBM\WebSphere\AppServer`, to OpenSSHD-style paths, e.g. `/C:/Program Files/IBM/WebSphere/AppServer`, so that your code can use Windows style paths. #### SUDO and INTERACTIVE_SUDO To use the __SUDO__ connection type, the `/etc/sudoers` configuration will have to be set up in such a way that the user configured with the connection option __username__ can execute the commands below as the user configured with the connection option __sudoUsername__. The arguments passed to these commands depend on the exact usage of the Overthere connection. Check the `INFO` messages on the `com.xebialabs.overthere.ssh.SshConnection` category to see what commands get executed. * `chmod` * `cp` * `ls` * `mkdir` * `mv` * `rm` * `rmdir` * `tar` * Any other command that you want to execute. The commands mentioned above must be configured with the __NOPASSWD__ setting in the `/etc/sudoers` file. Otherwise you will have to use the __INTERACTIVE_SUDO__ connection type. When the __INTERACTIVE_SUDO__ connection type is used, every line of the output will be matched against the regular expression configured with the __sudoPasswordPromptRegex__ connection option. If a match is found, the value of the __password__ connection option is sent. ### Troubleshooting SSH connections This section lists a number of common configuration errors that can occur when using Overthere with SSH. If you run into other connectivity issues when using Overthere, pease let us know by [creating a ticket](https://github.com/xebialabs/overthere/issues) or by [sending us a pull request](https://github.com/xebialabs/overthere/pulls). #### Cannot start a process on an SSH server because the server disconnects immediately. If the terminal type requested using the [__allocatePty__](#ssh_allocatePty) connection option or the [__allocateDefaultPty__](#ssh_allocateDefaultPty) connection option is not recognized by the SSH server, the connection will be dropped. Specifically, the `dummy` terminal type configured by [__allocateDefaultPty__] connection option, will cause OpenSSH on AIX and WinSSHD to drop the connection. Try a safe terminal type such as `vt220` instead. To verify the behaviour of your SSH server with respect to pty allocation, you can manually execute the
ssh
command with the `-T` (disable pty allocation) or `-t` (force pty allocation) flags.
#### Command executed using SUDO or INTERACTIVE_SUDO fails with the message `sudo: sorry, you must have a tty to run sudo`
The `sudo` command requires a tty to run. Set the [__allocatePty__](#ssh_allocatePty) connection option or the [__allocateDefaultPty__](#ssh_allocateDefaultPty) connection option to ask the SSH server allocate a pty.
#### Command executed using SUDO or INTERACTIVE_SUDO appears to hang.
This may be caused by the sudo command waiting for the user to enter his password to confirm his identity. There are two ways to solve this:
1. Use the [`NOPASSWD`](http://www.gratisoft.us/sudo/sudoers.man.html#nopasswd_and_passwd) tag in your `/etc/sudoers` file.
2. Use the [__INTERACTIVE_SUDO__](#ssh_host_setup_interactive_sudo) connection type instead of the [__SUDO__](ssh_host_setup_sudo) connection type.
3. If you are already using the __INTERACTIVE_SUDO__ connection type and you still get this error, please verify that you have correctly configured the [__sudoPasswordPromptRegex__](#ssh_sudoPasswordPromptRegex) option. If you have trouble determining the proper value for the __sudoPasswordPromptRegex__ connection option, set the log level for the `com.xebialabs.overthere.ssh.SshElevatedPasswordHandlingStream` category to `TRACE` and examine the output.
### SSH connection options
The SSH protocol implementation of Overthere defines a number of additional connection options, in addition to the [common connection options](#common_connection_options).
connectionType | Specifies how the SSH protocol is used. One of the following values must be set:
port specifies the port on which the SSH server listens. The default value for is 22 for all
connection types. |
---|---|
suUsername | The username of the user that can manipulate the files that need to be manipulated and that can execute the commands that need to be executed.
N.B.: This connection option is only applicable for the SU connection type. |
suPassword | The password of the user that can manipulate the files that need to be manipulated and that can execute the commands that need to be executed.
N.B.: This connection option is only applicable for the SU connection type. |
sudoUsername | The username of the user that can manipulate the files that need to be manipulated and that can execute the commands that need to be executed.
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
privateKey | The RSA private key as String to use when connecting to the remote host. When this connection option is specified, the password and privateKeyFile connection options are ignored. |
privateKeyFile | The RSA private key file to use when connecting to the remote host. When this connection option is specified, the password connection option is ignored. |
passphrase | The passphrase to unlock the RSA private key file specified with the privateKeyFile connection option. If this connection option is not specified, the RSA private key file must have an empty passphrase. |
allocateDefaultPty | If set to true , the SSH server is requested to allocate a default pty for the process, as if the allocatePty option were set to the value dummy:80:24:0:0 . The default value is false .
N.B.: This connection option has been deprecated in favour of the allocatePty connection option because it allows the user to specify _what_ pty is allocated. |
allocatePty | If set, the SSH server is requested to allocate a pty (pseudo terminal) for the process with
the setting specified by this option. The format is TERM:COLS:ROWS:WIDTH:HEIGHT , e.g. xterm:80:24:0:0 . If set, this option
overrides the allocateDefaultPty option.If the INTERACTIVE_SUDO connection type is used, the default value is vt220:80:24:0:0 . Otherwise the default is to not allocate a pty.All su and sudo implementations require a pty to be allocated for when displaying a password prompt, some sudo implementations even require it when not displaying a password prompt.Some SSH server implementations (notably OpenSSH on AIX 5.3) close the connection when an unknown one is allocated. |
heartbeatInterval | Specify an interval to send keep-alives packets. Default is 0 (no keep-alive). |
interactiveKeyboardAuthRegex | The regular expression to look for in keyboard-interactive prompts before sending the password. The default value is .*Password:[ ]? . When the SSH server is configured to not allow password authentication but is configured to allow keyboard-interactive authentication using passwords, Overthere will compare the interactive-keyboard prompt against this regular expression and send the value of the password option when they match. The default value is .*Password:[ ]? |
openShellBeforeExecute | If set to true , Overthere will open and close a shell immediately before executing a command on an ssh host. This is useful when the connecting user does not yet have a homedir, but this is created for him on the fly on the host. A setup commonly seen when user management is done through LDAP. |
suCommandPrefix | The command to prefix to the command to be executed to execute it as suUsername. The string {0} is replaced with the
value of suUsername. The default value is su - {0} -c .
N.B.: This connection option is only applicable for the SU connection type. |
suOverrideUmask | If set to true , Overthere will explicitly change the permissions with chmod -R go+rX after uploading a file or directory with
scp. The default value is true .
N.B.: This connection option is only applicable for the SU connection type. |
suPasswordPromptRegex | The regular expression to be used when looking for su password prompts. When the connection type is set to INTERACTIVE_SUDO, Overthere will look for strings that match this regular expression in the first line of the output of a command, and send the password if a match occurs. The default value is .*[Pp]assword.*:
N.B.: This connection option is only applicable for the SU connection type. |
suPreserveAttributesOnCopyFromTempFile | If set to true , files are copied from the connection temporary directory using the -p flag to the cp command. The default value is false .
N.B.: This connection option is only applicable for the SU connection type. |
suPreserveAttributesOnCopyToTempFile | If set to true , files are copied to the connection temporary directory using the -p flag to the cp command. The default value is false .
N.B.: This connection option is only applicable for the SU connection type. |
suQuoteCommand | If set to true , the original command is added as one argument to the prefix configured with the suCommandPrefix connection option. This has the result of quoting the original command, which is needed for commands like su . Compare su -u privilegeduser start server1 to su privilegeduser -c 'start server1' . The default value is true .
N.B.: This connection option is only applicable for the SU connection type. |
sudoCommandPrefix | The command to prefix to the command to be executed to execute it as sudoUsername. The string {0} is replaced with the
value of sudoUsername. The default value is sudo -u {0} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoOverrideUmask | If set to true , Overthere will explicitly change the permissions with chmod -R go+rX after uploading a file or directory with
scp. The default value is true .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoPasswordPromptRegex | The regular expression to be used when looking for sudo password prompts. When the connection type is set to INTERACTIVE_SUDO, Overthere will look for strings that match this regular expression in the first line of the output of a command, and send the password if a match occurs. The default value is .*[Pp]assword.*:
N.B.: This connection option is only applicable for the INTERACTIVE_SUDO connection type. |
sudoPreserveAttributesOnCopyFromTempFile | If set to true , files are copied from the connection temporary directory using the -p flag to the cp command. The default value is false .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoPreserveAttributesOnCopyToTempFile | If set to true , files are copied to the connection temporary directory using the -p flag to the cp command. The default value is false .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoQuoteCommand | If set to true , the original command is added as one argument to the prefix configured with the sudoCommandPrefix connection option. This has the result of quoting the original command, which is needed for commands like su . Compare sudo -u privilegeduser start server1 to su privilegeduser -c 'start server1' . The default value is false .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
deleteDirectoryCommand | The command to be used when deleting a directory. The string {0} is replaced with the value of the path of the directory to be deleted. The default value is rmdir {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
deleteFileCommand | The command to be used when deleting a file. The string {0} is replaced with the value of the path of the file to be deleted. The default value is rm -f {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
deleteRecursivelyCommand | The command to be used when deleting a directory recursively. The string {0} is replaced with the value of the path of the directory to be deleted. The default value is rm -rf {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
getFileInfoCommand | The command to be used when getting the metadata of a file/directory. The string {0} is replaced with the value of the path of the file/directory. The default value is ls -ld {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
listFilesCommand | The command to be used when listing the contents of a directory. The string {0} is replaced with the value of the path of the directory to be listed. The default value is ls -a1 {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
mkdirCommand | The command to be used when creating a directory. The string {0} is replaced with the value of the path of the directory to be created. The default value is mkdir {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
mkdirsCommand | The command to be used when creating a directory tree. The string {0} is replaced with the value of the path of the directory tree to be created. The default value is mkdir -p {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
renameToCommand | The command to be used when renaming a file/directory. The string {0} is replaced with the value of the path of the file/directory to be renamed. The string {1} is replaced with the value of the new name. The default value is mv {0} {1} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
setExecutableCommand | The command to be used when making a file executable. The string {0} is replaced with the value of the path of the file/directory affected. The default value is chmod a+x {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
setNotExecutableCommand | The command to be used when making a file non-executable. The string {0} is replaced with the value of the path of the file/directory affected. The default value is chmod a-x {0} .
N.B.: This connection option is only applicable for the SCP, SUDO and INTERACTIVE_SUDO connection types. |
sudoTempMkdirCommand | The command to be used when creating a temporary directory as a sudo user. The directory needs to be read/writeable for both the connecting and the sudo user. The string {0} is replaced with the value of the path of the directory to be created. The default value is mkdir -m 1777 {0} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoTempMkdirsCommand | The command to be used when creating a temporary directory tree as a sudo user. The directory tree needs to be read/writeable for both the connecting and the sudo user. The string {0} is replaced with the value of the path of the directory to be created. The default value is mkdir -p -m 1777 {0} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoCopyFromTempFileCommand | The command to be used when copying files/directories from the connection temporary directory as the sudo user. The string {0} is replaced with the value of the path of the file/directory being copied. The string {1} is replaced with the value of the target path. The default value is cp -pr {0} {1} if sudoPreserveAttributesOnCopyFromTempFile is set to true, otherwise the default value is cp -r {0} {1} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoOverrideUmaskCommand | The command to be used when setting the umask before copying a file/directory from, or after copying it to the connection temporary directory. This command ensures that the sudo user has read (and/or execute) rights for the copied file/directory. The string {0} is replaced with the value of the file/directory being copied. The default value is chmod -R go+rX {0} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
sudoCopyToTempFileCommand | The command to be used when copying files/directories to the connection temporary directory as the sudo user. The string {0} is replaced with the value of the path of the file/directory being copied. The string {1} is replaced with the value of the target path. The default value is cp -pr {0} {1} if sudoPreserveAttributesOnCopyToTempFile is set to true, otherwise the default value is cp -r {0} {1} .
N.B.: This connection option is only applicable for the SUDO and INTERACTIVE_SUDO connection types. |
localAddress | The address to use on the local machine as the source address of the connection. This property is optional and mainly useful on systems with more than one address. The default behaviour is to let the OS decide. |
localPort | The port to use on the local machine as the source port of the connection. This property is optional and to be used in combination with the localAddress property. The default behaviour is to let the OS pick a free port. |
transportTimeoutMillis | The number of milliseconds Overthere will wait for exchanging SSH keys and to authenticate user. The default value is 30000 , i.e.
30s timeout. |
WSMAN/HOST
, for which an SPN should be configured automatically when you [configure WinRM for a remote host](#smb_cifs_host_setup_winrm).
If that was not configured correctly, if you have overridden the default SPN for which a ticket is requested through the [__winrmKerberosAddPortToSpn__](#smb_cifs_winrmKerberosAddPortToSpn) or the [__winrmKerberosUseHttpSpn__](#smb_cifs_winrmKerberosUseHttpSpn) connection properties, or if you are running an older version of Overthere, you will have configure the service principal names manually.
The parameter [__winrmUseCanonicalHostname__](#smb_cifs_winrmUseCanonicalHostname) can be used to use the remote host FQDN provided by the DNS.
This can be achieved by invoking the setspn command, as an Administrator, on any host in the domain, as follows:
setspn -A PROTOCOL/ADDRESS:PORT WINDOWS-HOSTwhere: * `PROTOCOL` is either `WSMAN` (default) or `HTTP` (if [__winrmKerberosUseHttpSpn__](#smb_cifs_winrmKerberosUseHttpSpn) has been set to `true`). * `ADDRESS` is the [__address__](#address) used to connect to the remote host, * `PORT` (optional) is the [__port__](#port) used to connect to the remote host (usually 5985 or 5986, only necessary if [__winrmKerberosAddPortToSpn__](#smb_cifs_winrmKerberosAddPortToSpn) has been set to `true`), and * `WINDOWS-HOST` is the short Windows hostname of the remote host. Some other useful commands: * List all service principal names configured for the domain: `setspn -Q */*` *
本源码包内暂不包含可直接显示的源代码文件,请下载源码包。