Why are Carriers now using APIs to validate Shipping Data, and How will this impact your Shipping Processes?
Everyone in business is familiar with the concept of technology evolving to improve productivity and customer service outcomes. Software has played a key role in this process and will continue to so on an ever-increasing basis in the future. In the area of shipping software platforms, simple and easy integration between systems and providers is a key strategy for making shipping easier and quicker for all market participants.
What is an API?
API, or Application Programming Interface is a methodology of interfacing many different software systems via a standard protocol thereby allowing for relatively quick, independent and cost effective networking of many different market participants and systems.
How does an API apply to the Freight Industry?
Because of the standardisation of connections, many major freight carriers have now taken the step to develop APIs for the purpose of allowing their partners and customers to interface with them to transfer data to and from each other, often in real time, or near real time. APIs enable carriers to standardise the data coming in to and out of their systems back to their customers.
What did Carriers use prior to API connections?
Electronic Data Interchange (EDI) is the method by which customers would typically use to submit manifested data to carriers on a daily basis. This is what we refer to as the legacy system before the introduction of API. Without doing a deep dive into the technical details, EDI could be described as the carriers providing a file template in CSV file format or XML.
Customers or software platform partners would generate these files with the manifested data and send the data off to the carriers via email or ftp (File transfer protocol). The carriers would then import this data into their billing and sortation systems for their operational purposes. While it is possible for carrier to generate similar standardised file templates for sending delivery event or Proof of Delivery (POD) data back to customers, that is not typical or widespread with carriers preferring that the vast majority of the customers get their delivery event data from the carrier websites one shipment at a time – manually.
What are the direct benefits to the Carriers moving to API for Data transfer purposes?
Putting aside indirect benefits of standardisation, probably the greatest benefit for carriers introducing API is that they have the ability to validate incoming shipping data from the shipper’s manifest. The process of validating shipping data is highly beneficial to the carriers as it prevents carriers from receiving invalid or incorrect data. This invalid data can cause issues with the process of successfully importing data into the carrier’s sortation and billing systems. Carriers then must spend time and money correcting the data before it can be imported.
Examples of data that might be considered as invalid or unhelpful are:
- Incorrectly formatted phone and email addresses for the receivers.
- Street addresses that are too long or full of abbreviations and incorrect symbols.
- Invalid post codes.
- Invalid spelling of suburbs or mismatched town name and post codes.
- Incorrect package descriptions.
- Nonsensical package dimensions.
- Shippers trying to send over dimensional or prohibited packages such as DG items, or packages packed onto pallet and skids where the carrier strictly prohibits these items.
As you can see, and especially in the case where shippers are trying to ship prohibited items, or packages that are overweight or sized, the carriers say that preventing these errors helps greatly to improve their productivity, reduce operational costs associated with trying to return or ship items that are not meant to be in their network to start with. In fact, so adamant are the carriers on these points that one head of IT at a major national carrier said to me that “the days of them accepting crap data from clients has gone”!
What are the benefits to the Shipper moving to an API Data transfer?
Of course, the benefit of better data flow is not experienced by the carrier alone – the shippers also benefit greatly from getting the data right the first time. It means a better and more consistent delivery service and also means that errors are picked up before the shipment is sent and that prevents errors and late deliveries due to labelling issues.
As mentioned above, the introduction of API based connections also allows for more cost-effective and an easier method of transferring data from the carrier to the shipper around delivery events and PODs. This is of huge benefit to the shipper, as it allows the shipper to potentially receive real time delivery event data on successful deliveries, on delays, as well as POD data allowing the shippers to pass on to their receiver’s delivery or transit data automatically, thereby improving the transparency of carrier performance through the whole delivery chain.
What are the impacts on the Shipper from the process of validating Shipping Data via API?
Now that we have discussed the benefits to both the carriers and the shippers that will flow from a better level of data interchange, there are changes that shippers will have to accommodate to their current shipping processes because of the requirement to validate the shipping data before they ship the consignment.
Essentially there are two opportunities for the carrier to validate the shipping data. Most of the major carriers perform this function when the shippers call for the label. The majority of carriers provide the shipper with the label, (again, because the carrier wants to have a uniform and valid label on the freight). Therefore, before the label is produced the carriers will validate the data. If the data does not conform to their requirements, then the label will not be provided, and an error message will be sent that allows the shipper to modify, or correct the data, and then receive the address label.
The other opportunity for validation of the data is when the shipper manifests the shipments, normally at the end of the day when the truck arrives to collect the goods. This is far less common than the previous scenario, but some carriers could choose this path if they do not provide the address label. That is, the shipper creates the label based upon their own data and prints it accordingly. Then, once the truck comes to collect the cargo the shipper must manifest, or close the shipments for the day, or for that truck, and at this point the API will validate. This second scenario is closer to the legacy process before the API introduction, but the adding of shipment validation brings significant challenges to the shipper and the carrier.
What are the Pros and Cons for validating Shipping Data when creating the Label?
Under this methodology the shippers knows immediately that there is an issue with the data and they can correct the error then and there and produce the correct label with validated data and no longer have to worry about whether or not the shipment is valid. This is because the carrier now considers that the shipment has been sent, and it has been manifested. In order to validate the shipment data, the shipper must send the data to the carrier, and therefore the carrier considers the shipment as sent.
The con with this process, and where some high volume shippers may find it challenging, is that if you wish to amend the shipment then you may need to delete the shipment and create a new one. This is because all but the most sophisticated carriers have an API that supports amendments to existing shipments and some don’t have APIs that cover cancellations. This means that if you delete the shipment in your TMS after printing the label, in some cases, you may have to contact the carrier by phone or email to cancel the shipment at their end. This is because they may consider the shipment as live and therefore you maybe charged even though the shipment has not be dispatched.
The pro for not validating the data until you manifest is that if you wish to cancel the shipment or amend it you can do so without notifying the carrier because the carrier is unaware of the shipment at this stage. This sounds appealing as shippers are familiar with this, but we believe that its actually far less convenient than validating when printing the label, because now you have prepared all your shipments and the truck arrives and you start validating the shipment data.
In order for carriers to identify any invalid data and which shipment that error belongs to with a way of amending the shipment, they need to validate each shipment that you manifest, potentially one at a time. Or if they process the manifest in bulk, rejecting the entire manifest because of an error in one or more shipments.
In some cases, the carriers cannot identify the error for a particular shipment hence the reason they process each shipment one at a time when manifesting. As you can imagine this process of correcting errors and re submitting manifests all at the end of the day, after the packages have been potentially packed onto consolidated pallets is not convenient and far more complicated than the relatively simple process of correcting the error when the shipment was first lodged and the label printed.
In the most sophisticated option the software maybe able to write the errors to a file and the shipper can go back and correct only the shipments that had errors, but this does mean that the process of manifesting or closing the shipments will be time consuming for a large number of shipments, reinforcing our opinion that it’s better to validate the data when first generating the label and then dealing with the exceptions such as the need to amend or delete a shipment and contacting the carrier.
Conclusion
While at first shippers may be reluctant to see the move to the validation of data prior to shipping as a positive change due to the fact that they can no longer just send any old data through, (they must take the time and accountability to clean and validate the shipping data), they will be rewarded with an immediate response from the carrier with far greater flow of useful data like real time and up to date price estimates, as well as real time tracking event data. By providing error free data, carrier delivery performance will improve, and this benefits all parties. A change of shipping practices is required, and this may initially feel as though it’s time consuming, but in logistics as in life, it’s always more efficient to do it once and do it right.
If you have some questions about API-based connections with carriers or would like to learn about our TMS system that supports API carriers, please contact us to discuss.