HTTPS: Difference between revisions

From Rosetta Code
Content added Content deleted
Line 482: Line 482:


=={{header|NewLISP}}==
=={{header|NewLISP}}==
<lang=newlisp>(! "curl https://sourceforge.net")</lang>
<lang newlisp>(! "curl https://sourceforge.net")</lang>


=={{header|Nim}}==
=={{header|Nim}}==

Revision as of 21:40, 29 March 2018

Task
HTTPS
You are encouraged to solve this task according to the task description, using any language you may know.
Task

Print an HTTPS URL's content to the console. Checking the host certificate for validity is recommended.

The client should not authenticate itself to the server — the webpage https://sourceforge.net/ supports that access policy — as that is the subject of other tasks.

Readers may wish to contrast with the HTTP Request task, and also the task on HTTPS request with authentication.

Ada

Library: AWS

Exactly the same as the HTTP task, assuming you compiled AWS with openssl support. <lang ada> with AWS.Client; with AWS.Response; with Ada.Text_IO; use Ada.Text_IO; procedure GetHttps is begin

  Put_Line (AWS.Response.Message_Body (AWS.Client.Get (
     URL => "https://sourceforge.net/")));

end GetHttps; </lang>

AutoHotkey

Library: wininet

<lang AutoHotkey> URL  := "https://sourceforge.net/" WININET_Init() msgbox % html := UrlGetContents(URL) WININET_UnInit() return

  1. include urlgetcontents.ahk
  2. include wininet.ahk

</lang>

Batch File

<lang batch>

Must have curl.exe

curl.exe -k -s -L https://sourceforge.net/ </lang>

C

Library: libcurl

<lang c>

  1. include <stdio.h>
  2. include <stdlib.h>
  3. include <curl/curl.h>

int main(void) {

       CURL *curl;
       char buffer[CURL_ERROR_SIZE];
       if ((curl = curl_easy_init()) != NULL) {
               curl_easy_setopt(curl, CURLOPT_URL, "https://sourceforge.net/");
               curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
               curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, buffer);
               if (curl_easy_perform(curl) != CURLE_OK) {
                       fprintf(stderr, "%s\n", buffer);
                       return EXIT_FAILURE;
               }
               curl_easy_cleanup(curl);
       }
       return EXIT_SUCCESS;

} </lang>

C#

Works with: C sharp version 3.0

<lang csharp> using System; using System.Net;

class Program {

   static void Main(string[] args)
   {
       var client = new WebClient();
       var data = client.DownloadString("https://www.google.com");
       Console.WriteLine(data);
   }

} </lang>

This does not work for urls requiring a secure (SSL) connection.

Clojure

Using the duck-streams as a convenient wrapper for Java's networking classes, grabbing the contents of an HTTPS URL is as easy as:

<lang clojure> (use '[clojure.contrib.duck-streams :only (slurp*)]) (print (slurp* "https://sourceforge.net")) </lang>

The usual Java mechanisms can be used to manage acceptance of SSL certificates if required.

Works with: Clojure version 1.2

<lang clojure> (print (slurp "https://sourceforge.net")) </lang>

Common Lisp

Library: DRAKMA

First grabbing the entire body as a string, and then by pulling from a stream. This is the same code as in HTTP Request; drakma:http-request supports SSL.

<lang lisp> (defun wget-drakma-string (url &optional (out *standard-output*))

 "Grab the body as a string, and write it to out."
 (write-string (drakma:http-request url) out))

(defun wget-drakma-stream (url &optional (out *standard-output*))

 "Grab the body as a stream, and write it to out."
 (loop with body = (drakma:http-request url :want-stream t)
       for line = (read-line body nil nil)
       while line do (write-line line)
       finally (close body)))
Use

(wget-drakma-stream "https://sourceforge.net") </lang>

Delphi

Library: OpenSSL

<lang Delphi> program ShowHTTPS;

{$APPTYPE CONSOLE}

uses IdHttp, IdSSLOpenSSL;

var

 s: string;
 lHTTP: TIdHTTP;

begin

 lHTTP := TIdHTTP.Create(nil);
 try
   lHTTP.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(lHTTP);
   lHTTP.HandleRedirects := True;
   s := lHTTP.Get('https://sourceforge.net/');
   Writeln(s);
 finally
   lHTTP.Free;
 end;

end. </lang>

EchoLisp

file->string usage: the server must allow cross-domain access, or a browser add-on like cors-everywhere must be installed to bypass cross-domain checking. <lang scheme>

asynchronous call back definition

(define (success name text) (writeln 'Loaded name) (writeln text))

(file->string success "https:/sourceforge.net") </lang>

Erlang

Synchronous

<lang erlang> -module(main). -export([main/1]).

main([Url|[]]) ->

  inets:start(),
  ssl:start(),
  case http:request(get, {URL, []}, [{ssl,[{verify,0}]}], []) of
      {ok, {_V, _H, Body}} -> io:fwrite("~p~n",[Body]);
      {error, Res} -> io:fwrite("~p~n", [Res])
  end.

</lang>

Asynchronous

<lang erlang> -module(main). -export([main/1]).

main([Url|[]]) ->

  inets:start(),
  ssl:start(),
  http:request(get, {Url, [] }, [{ssl,[{verify,0}]}], [{sync, false}]),
  receive
      {http, {_ReqId, Res}} -> io:fwrite("~p~n",[Res]);
      _Any -> io:fwrite("Error: ~p~n",[_Any])
      after 10000 -> io:fwrite("Timed out.~n",[])
  end.

</lang>

Using it <lang erlang> |escript ./req.erl https://sourceforge.net/ </lang>

Fortran

There is no such network library for communicating with a HTTPS server in fortran. Use appropriate tools (eg. simple node.js snippet) <lang fortran> program https_example

  implicit none
  character (len=:), allocatable :: code
  character (len=:), allocatable :: command
  logical:: waitForProcess
  ! execute Node.js code
  code = "var https = require('https'); &
  https.get('https://sourceforge.net/', function(res) {&
  console.log('statusCode: ', res.statusCode);&
  console.log('Is authorized:' + res.socket.authorized);&
  console.log(res.socket.getPeerCertificate());&
  res.on('data', function(d) {process.stdout.write(d);});});"
  command = 'node -e "' // code // '"'
  call execute_command_line (command, wait=waitForProcess)

end program https_example </lang>

F#

The underlying .NET classes handle secure web connections the same way they manage insecure connections. <lang fsharp>

  1. light

let wget (url : string) =

   let c = new System.Net.WebClient()
   c.DownloadString(url)

</lang>

Frink

<lang Frink> print[read["https://sourceforge.net/"] </lang>

Go

<lang go> package main

import (

   "io"
   "log"
   "net/http"
   "os"

)

func main() {

   r, err := http.Get("https://sourceforge.net/")
   if err != nil {
       log.Fatalln(err)
   }
   io.Copy(os.Stdout, r.Body)

} </lang>

Groovy

<lang groovy> new URL("https://sourceforge.net").eachLine { println it } </lang>

Haskell

Library: http-conduit
Works with: GHC version 7.4.1

This is just the example from Network.HTTP.Conduit, with the http URL replaced with an https one, since http-conduit natively supports https without needing any additional work.

<lang haskell>#!/usr/bin/runhaskell

import Network.HTTP.Conduit import qualified Data.ByteString.Lazy as L import Network (withSocketsDo)

main = withSocketsDo

    $ simpleHttp "https://sourceforge.net/" >>= L.putStr</lang>

Icon and Unicon

<lang unicon># Requires Unicon version 13 procedure main(arglist)

   url := (\arglist[1] | "https://sourceforge.net/")
   w := open(url, "m-") | stop("Cannot open " || url)
   while write(read(w))
   close(w)

end</lang>

Output:
prompt$ unicon -s https.icn -x | head -n2
<!doctype html>
<!-- Server: sfs-consume-15 -->

Ioke

Translation of: Java

<lang ioke> connection = URL new("https://sourceforge.net") openConnection scanner = Scanner new(connection getInputStream)

while(scanner hasNext,

 scanner next println

) </lang>

J

Using gethttp from Web Scraping

<lang j>

  #page=: gethttp'https://sourceforge.net'

0

  #page=: '--no-check-certificate' gethttp'https://sourceforge.net'

900 </lang>

(We can not load the example page using https unless we disable certificate checking. The numbers are the number of characters retrieved.)

Java

Additional certificate information is available through the javax.net.ssl.HttpsURLConnection interface. <lang Java> URL url = new URL("https://sourceforge.net"); HttpsURLConnection connection = (HttpsURLConnection) url.openConnection(); Scanner scanner = new Scanner(connection.getInputStream());

while (scanner.hasNext()) {

   System.out.println(scanner.next());

} </lang>

JavaScript

<lang JavaScript> (function(url,callback){//on some browsers you can check certificate information. xhr=new XMLHttpRequest(); xhr.open('GET',url,true); xhr.onreadystatechange=function(){if(xhr.readyState==xhr.DONE){callback(xhr)}}; xhr.send(); })('https://sourceforge.net',function(xhr){console.log(xhr.response)}) </lang>

Julia

<lang julia># v0.6.0

using Requests

str = readstring(get("https://sourceforge.net/"))</lang>

Kotlin

<lang scala>// version 1.1.2 import java.net.URL import javax.net.ssl.HttpsURLConnection import java.io.InputStreamReader import java.util.Scanner

fun main(args: Array<String>) {

   val url = URL("https://en.wikipedia.org/wiki/Main_Page")
   val connection = url.openConnection() as HttpsURLConnection
   val isr = InputStreamReader(connection.inputStream)
   val sc = Scanner(isr)
   while (sc.hasNextLine()) println(sc.nextLine())
   sc.close()

}</lang>

Lasso

<lang Lasso>local(x = curl('https://sourceforge.net')) local(y = #x->result)

  1. y->asString</lang>

If a site with an invalid SSL Cert is encountered the curl type throws the following error:

Output:
FAILURE: 60 Peer certificate cannot be authenticated with given CA certificates

Lingo

  • Windows:
Library: Curl Xtra

<lang lingo>ch = xtra("Curl").new() CURLOPT_URL = 10002 ch.setOption(CURLOPT_URL, "https://sourceforge.net") res = ch.exec(1) if integerP(res) then

 put "Error:" && curl_error(res)

else

 put "Result:" && res.readRawString(res.length)

end if -- "Result: <!doctype html> ..."</lang>

  • Mac OS X:
Library: Shell Xtra

<lang lingo>sx = xtra("Shell").new() put sx.shell_cmd("curl https://sourceforge.net")</lang>

LiveCode

Blocking version<lang LiveCode>libURLSetSSLVerification true --check cert get URL "https://sourceforge.net/"</lang> Non-blocking version, execute getWebResource <lang LiveCode>on myUrlDownloadFinished

   get URL "https://sourceforge.net/" -- this will now fetch a locally cached copy
   put it

end myUrlDownloadFinished

command getWebResource

   libURLFollowHttpRedirects true
   libURLSetSSLVerification true  --check cert
   load URL "https://sourceforge.net/" with message "myUrlDownloadFinished"

end getWebResource</lang>

LSL

Virtually identical to the HTTP Task.

To test it yourself; rez a box on the ground, and add the following as a New Script. <lang LSL>string sURL = "https://SourceForge.Net/"; key kHttpRequestId; default { state_entry() { kHttpRequestId = llHTTPRequest(sURL, [], ""); } http_response(key kRequestId, integer iStatus, list lMetaData, string sBody) { if(kRequestId==kHttpRequestId) { llOwnerSay("Status="+(string)iStatus); integer x = 0; for(x=0 ; x<llGetListLength(lMetaData) ; x++) { llOwnerSay("llList2String(lMetaData, "+(string)x+")="+llList2String(lMetaData, x)); } list lBody = llParseString2List(sBody, ["\n"], []); for(x=0 ; x<llGetListLength(lBody) ; x++) { llOwnerSay("llList2String(lBody, "+(string)x+")="+llList2String(lBody, x)); } } } }</lang> Output:

Status=200
llList2String(lMetaData, 0)=0
llList2String(lMetaData, 1)=2048
llList2String(lBody, 0)=<!doctype html>
llList2String(lBody, 1)=<!-- Server: sfs-consume-7 -->
llList2String(lBody, 2)=<!--[if lt IE 7 ]> <html lang="en" class="no-js ie6" > <![endif]-->
llList2String(lBody, 3)=<!--[if IE 7 ]>    <html lang="en" class="no-js ie7" > <![endif]-->
llList2String(lBody, 4)=<!--[if IE 8 ]>    <html lang="en" class="no-js ie8" > <![endif]-->
llList2String(lBody, 5)=<!--[if IE 9 ]>    <html lang="en" class="no-js ie9" > <![endif]-->
llList2String(lBody, 6)=<!--[if (gt IE 9)|!(IE)]>--> <html lang="en" class="no-js"> <!--<![endif]-->
llList2String(lBody, 7)=    <head>
llList2String(lBody, 8)=        <meta charset="utf-8">
llList2String(lBody, 9)=        
llList2String(lBody, 10)=        <meta id="webtracker" name="webtracker" content='{"event_id": "ea71f064-ca28-11e1-98cc-0019b9f0e8fc"}'>
llList2String(lBody, 11)=        <meta name="description" content="Free, secure and fast downloads from the largest Open Source applications and software directory - SourceForge.net">
llList2String(lBody, 12)=        <meta name="keywords" content="Open Source, Open Source Software, Development, Community, Source Code, Secure,  Downloads, Free Software">
llList2String(lBody, 13)=<meta name="msvalidate.01" content="0279349BB9CF7ACA882F86F29C50D3EA" />
llList2String(lBody, 14)=        <meta name="viewport" content="width=device-width, initial-scale=1.0">
llList2String(lBody, 15)=        <title>SourceForge - Download, Develop and Publish Free Open Source Software</title>
llList2String(lBody, 16)=        <link rel="shortcut icon" href="http://a.fsdn.com/con/img/sftheme/favicon.ico">
...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...   ...

Maple

<lang Maple> content := URL:-Get( "https://www.google.ca/" ); </lang>

Mathematica / Wolfram Language

Straight forward "Import" task. More complicated secure web access can be done using J/Link; essentially a link to Java API. <lang Mathematica> content=Import["https://sourceforge.net", "HTML"] </lang>

MATLAB / Octave

<lang MATLAB>s=urlread('https://sourceforge.net/')</lang>

Nemerle

This example is essentially identical to the HTTP task because the WebClient object can be used with http:, https:, ftp: and file: uri's. <lang Nemerle>using System; using System.Console; using System.Net; using System.IO;

module HTTP {

   Main() : void
   {
       def wc = WebClient();
       def myStream = wc.OpenRead(https://sourceforge.com);
       def sr = StreamReader(myStream);
       
       WriteLine(sr.ReadToEnd());
       myStream.Close()
   }

}</lang>

NewLISP

<lang newlisp>(! "curl https://sourceforge.net")</lang>

Nim

Library: OpenSSL

Compile with nim c -d:ssl httpsClient.nim: <lang nim>import httpclient

var client = newHttpClient() echo client.getContent("https://sourceforge.net")</lang>

Objeck

<lang objeck> use HTTP;

class HttpsTest {

 function : Main(args : String[]) ~ Nil {
   client := HttpsClient->New();
   lines := client->Get("https://sourceforge.net");
   each(i : lines) {
     lines->Get(i)->As(String)->PrintLine();
   };
 }

} </lang>

Perl

Library: LWP

<lang perl> use strict; use LWP::UserAgent;

my $url = 'https://www.rosettacode.org'; my $response = LWP::UserAgent->new->get( $url );

$response->is_success or die "Failed to GET '$url': ", $response->status_line;

print $response->as_string; </lang>

Perl 6

Works with: Rakudo version 2017.09

There are several modules that provide HTTPS capability. WWW and HTTP::UserAgent are probably the most popular right now, but others exist.

<lang perl6>use WWW; say get 'https://sourceforge.net/';</lang> or <lang perl6>use HTTP::UserAgent; say HTTP::UserAgent.new.get: 'https://sourceforge.net/';</lang>

PHP

<lang php> echo file_get_contents('https://sourceforge.net'); </lang>

PicoLisp

PicoLisp has no functionality for communicating with a HTTPS server (only for the other direction), but it is easy to use an external tool <lang PicoLisp> (in '(curl "https://sourceforge.net") # Open a pipe to 'curl'

  (out NIL (echo)) )                  # Echo to standard output

</lang>

Pike

<lang pike> int main() {

   write("%s\n", Protocols.HTTP.get_url_data("https://sourceforge.net"));

} </lang>

PowerShell

<lang powershell> $wc = New-Object Net.WebClient $wc.DownloadString('https://sourceforge.net') </lang>

If the certificate could not be validated (untrusted, self-signed, expired), then an Exception is thrown with the message “The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.” so certificate validation is done automatically by the method.

Python

Python's urllib.request library, (urllib2 in Python2.x) has support for SSL if the interpreter's underlying httplib libraries were compiled with SSL support. By default this will be the enabled for default Python installations on most platforms.

Python 3.x: <lang Python> from urllib.request import urlopen print(urlopen('https://sourceforge.net/').read()) </lang>

(Python 2.x) <lang Python> from urllib2 import urlopen print urlopen('https://sourceforge.net/').read() </lang>

R

Library: RCurl
Library: XML

The basic idea is to use getURL (as with HTTP_Request), but with some extra parameters. <lang R>library(RCurl) webpage <- getURL("https://sourceforge.net/", .opts=list(followlocation=TRUE, ssl.verifyhost=FALSE, ssl.verifypeer=FALSE))</lang> In this case, the webpage output contains unprocessed characters, e.g. \" instead of " and \\ instead of \, so we need to process the markup.

<lang R> wp <- readLines(tc <- textConnection(webpage)) close(tc) </lang>

Finally, we parse the HTML and find the interesting bit.

<lang R> pagetree <- htmlTreeParse(wp) pagetree$children$html </lang>

Racket

<lang Racket>

  1. lang racket

(require net/url) (copy-port (get-pure-port (string->url "https://www.google.com")

                         #:redirections 100)
          (current-output-port))

</lang>

REALbasic

REALBasic provides an HTTPSecureSocket class for handling HTTPS connections. The 'Get' method of the HTTPSecureSocket is overloaded and can download data to a file or return data as a string, in both cases an optional timeout argument can be passed.

<lang REALbasic>

     Dim sock As New HTTPSecureSocket
     Print(sock.Get("https://sourceforge.net", 10))  //set the timeout period to 10 seconds.

</lang>

Ring

<lang ring> cStr= download("http://sourceforge.net/") see cStr + nl </lang>

RLaB

See HTTP#RLaB

Ruby

This solution doesn't use the open-uri convenience package that the HTTP Request#Ruby solution uses: the Net::HTTP object must be told to use SSL before the session is started.

<lang ruby> require 'net/https' require 'uri' require 'pp'

uri = URI.parse('https://sourceforge.net') http = Net::HTTP.new(uri.host,uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE

http.start do

 content = http.get(uri)
 p [content.code, content.message]
 pp content.to_hash
 puts content.body

end </lang>

outputs

["302", "Found"]
{"location"=>["http://sourceforge.net/"],
 "content-type"=>["text/html; charset=UTF-8"],
 "connection"=>["close"],
 "server"=>["nginx/0.7.60"],
 "date"=>["Sun, 30 Aug 2009 20:20:07 GMT"],
 "content-length"=>["229"],
 "set-cookie"=>
  ["sf.consume=89f65c6fadd222338b2f3de6f8e8a17b2c8f67c2gAJ9cQEoVQhfZXhwaXJlc3ECY2RhdGV0aW1lCmRhdGV0aW1lCnEDVQoH9gETAw4HAAAAhVJxBFUDX2lkcQVVIDEyOWI2MmVkOWMwMWYxYWZiYzE5Y2JhYzcwZDMxYTE4cQZVDl9hY2Nlc3NlZF90aW1lcQdHQdKmt73UN21VDl9jcmVhdGlvbl90aW1lcQhHQdKmt73UN2V1Lg==; expires=Tue, 19-Jan-2038 03:14:07 GMT; Path=/"]}
<html>
 <head>
  <title>302 Found</title>
 </head>
 <body>
  <h1>302 Found</h1>
  The resource was found at <a href="http://sourceforge.net/">http://sourceforge.net/</a>;
you should be redirected automatically.


 </body>
</html>

Scala

Library: Scala

<lang scala>import scala.io.Source

object HttpsTest extends App {

 System.setProperty("http.agent", "*")
  
 Source.fromURL("https://sourceforge.net").getLines.foreach(println)

}</lang>

Seed7

The library gethttps.s7i defines the function getHttps which uses the HTTPS protocol go get a file.

<lang seed7>$ include "seed7_05.s7i";

 include "gethttps.s7i";
 include "utf8.s7i";

const proc: main is func

 begin
   writeln(STD_UTF8_OUT, getHttps("sourceforge.net"));
 end func;</lang>

Sidef

<lang ruby>var lwp = require('LWP::UserAgent'); # LWP::Protocol::https is needed var url = 'https://rosettacode.org';

var ua = lwp.new(

   agent    => 'Mozilla/5.0',
   ssl_opts => Hash.new(verify_hostname => 1),

);

var resp = ua.get(url); resp.is_success || die "Failed to GET #{url.dump}: #{resp.status_line}"; print resp.decoded_content;</lang>

Swift

<lang Swift>import Foundation

// With https let request = NSURLRequest(URL: NSURL(string: "https://sourceforge.net")!)

NSURLConnection.sendAsynchronousRequest(request, queue: NSOperationQueue()) {res, data, err in // callback

   // data is binary
   if (data != nil) {
       let string = NSString(data: data!, encoding: NSUTF8StringEncoding)
       println(string)
   }

}

CFRunLoopRun() // dispatch</lang>

Tcl

Though Tcl's built-in http package does not understand SSL, it does support the registration of external handlers to accommodate additional protocols. This allows the use of the Tls package to supply the missing functionality with only a single line to complete the registration.

<lang tcl> package require http package require tls

  1. Tell the http package what to do with “https:” URLs.
  2. First argument is the protocol name, second the default port, and
  3. third the connection builder command

http::register "https" 443 ::tls::socket

  1. Make a secure connection, which is almost identical to normal
  2. connections except for the different protocol in the URL.

set token [http::geturl "https://sourceforge.net/"]

  1. Now as for conventional use of the “http” package

puts [http::data $token] http::cleanup $token </lang>

TUSCRIPT

<lang tuscript> $$ MODE TUSCRIPT SET DATEN = REQUEST ("https://sourceforge.net")

  • {daten}

</lang>

UNIX Shell

<lang bash> curl -k -s -L https://sourceforge.net/ </lang>

VBScript

Based on code at How to retrieve HTML web pages with VBScript via the Microsoft.XmlHttp object <lang vb> Option Explicit

Const sURL="https://sourceforge.net/"

Dim oHTTP Set oHTTP = CreateObject("Microsoft.XmlHTTP")

On Error Resume Next oHTTP.Open "GET", sURL, False oHTTP.Send "" If Err.Number = 0 Then

    WScript.Echo oHTTP.responseText

Else

    Wscript.Echo "error " & Err.Number & ": " & Err.Description

End If

Set oHTTP = Nothing </lang>

Visual Basic .NET

<lang vbnet> Imports System.Net

Dim client As WebClient = New WebClient() Dim content As String = client.DownloadString("https://sourceforge.net") Console.WriteLine(content) </lang>

zkl

Using the cURL library to do the heavy lifting: <lang zkl>zkl: var ZC=Import("zklCurl") zkl: var data=ZC().get("https://sourceforge.net") L(Data(36,265),826,0)</lang> get returns the text of the response along with two counts: the bytes of header in front of the html code and the byte count of stuff after the end of the page. So, if you wanted to look at the header:

zkl: data[0][0,data[1]).text
HTTP/1.1 200 OK
Server: nginx
Date: Sun, 23 Mar 2014 07:36:51 GMT
Content-Type: text/html; charset=utf-8
Connection: close
...

or some of the html:

zkl: data[0][data[1],200).text
<!doctype html>
<!-- Server: sfs-consume-8 -->

<!--[if lt IE 7 ]> <html lang="en" class="no-js ie6"> <![endif]-->
<!--[if IE 7 ]>    <html lang="en" class="no-js ie7"> <![endif]-->
<!--[if IE 8 ]>