Improper URL validation and incorrect responses returned for invalid URLS

[jnobr]

Current checks for URLs are not satisfactory to handle certain classes of erroroneous input.
Specific examples include:

• 'http://www0.cs\t.ucl.wak.uk/staff/w.langdon/\nhttp://www0.cs\t.ucl.wak.uk/staff/w.langdon/'
• 'http://www0.cs.ucl.wac.uk/staff/w.lwangdon/\nhttp://www0.cs.ucl.wac.uk/staff/w.lwangdon/'
• 'http://www0.cs.ucl.pac.uk/staff/w.langdon/hWBL_pape)arshtml\nhttp://www0.cs.ucl.pac.uk/staff/w.langdon/hWBL_pape)arshtml'
• 'http://www0.cs.ucl.pac.uk/staff.w.langdon/hWBL_pape)ars.html\nhttp://www0.cs.ucl.pac.uk/staff.w.langdon/hWBL_pape)ars.html'
• 'http://www0.cs.ucl.wac.uk/staff/w.langdon/\nhttp://www0.cs.ucl.wac.uk/staff/w.langdon/'
• 'http://www.cs.ucl.ac.uk/staff/w.langdon/\nhttp://www.cs.ucl.ac.uk/staff/w.langdon/'
• 'http://www0.cs.ucl.wac.uk/staff/w.lwangdon/\nhttp://www0.cs.ucl.wac.uk/staff/w.lwangdon/'

All of these inputs result in status code 301 instead of 404 or an exception. Additionally, the redirects provided are sometimes invalid URLs or URLs from the wrong domain.
Examples are:

• /staff/w.lwangdon/http:/www0.cs.ucl.wac.uk/staff/w.lwangdon/
• http://www0.cs.ucl.ac.uk/staff/w.langdon/http:/www.cs.ucl.ac.uk/staff/w.langdon/
• /staff/w.langdon/http:/www0.cs.ucl.wac.uk/staff/w.langdon/
• 'http://www0.cs.ucl.pac.uk/staff.w.langdon/hWBL_pape)ars.htmlhttp:/www0.cs.ucl.pac.uk/staff.w.langdon/hWBL_pape)ars.html'
• http://www0.cs.ucl.pac.uk/staff/w.langdon/hWBL_pape)arshtmlhttp:/www0.cs.ucl.pac.uk/staff/w.langdon/hWBL_pape)arshtml
• /staff/w.lwangdon/http:/www0.cs.ucl.wac.uk/staff/w.lwangdon/
• https://www.domainstore.co.uk/?domain=wak.uk
• https://a.nel.cloudflare.com/report/v4?s=fvWoO6qraNH%2FiPXeWC853sJ1hzyVn1WqVQmfVXlHN38YRPQWIH%2Bzio69yMkf3exJDXSRvDUNwXxPrVqYuj7EfMe7C7SY6Km0AQibvGgOrc7ou1VQmeeT32HFGuPpMl2wT9bf0kE%3D

This can be reproduced through a simple get request and by examining the response object.
request = httpcore.request("GET",{url})