Hi Charlie,
Quote:
Originally Posted by Silencio
And in today's world there is a lot of duplicate content posted on web pages especially in the form of RSS feeds and it doesn't seem to affect them to the extent that maybe it did, maybe because of the rise of RSS since the Florida Update. Just an observation.
|
If you are interested in the whole duplicate content thingy, have a look here. There is no simple answer to that one either
.
Duplicate Content and Near Duplicate Content Issues -Resources for SEO
Quote:
Originally Posted by Silencio
in my searches so far I have not found a good guide that says "If you want to get started with Datafeeds this is the best way to it from the Beginning" which is why I have been going around the block in this learning adventure.
|
The thing is that the subject is for beginners and requires serious technical knowledge to do it right. There are some scripts but they either produce something that is pretty much worthless or you are required to customize it so extensively that it is not easy again. But I see your point, its not spelled out very often. I am thinking about taking some of my responses here and craft something out of it to put on my site.
Quote:
Originally Posted by Silencio
I see a huge need for a revolution in the way datafeeds are produced and distributed and I know that you have been dealing with this issue much longer than I.
|
Amen to that. I can point you to forum posts that are 5 years old where you can hear me ***** and yell (after I made some very specific and relevant suggestions). Regarding the why. Not sure, not a priority with the networks, obviously.
Quote:
Originally Posted by Silencio
Carsten you advice is much appreciated! Thanks. I hope that through your help and resources I can find a good solution or two or three.....
|
My pleasure, I am glad that I could help.
Quote:
Originally Posted by Silencio
If you, with the knowledge that you have today, were to recommend to someone the best way to get started with Datafeeds, to get them started in the right direction, what would you suggest to them in a step by step approach?
|
- Get at least some programming skills and also some experience with database software (MySQL or MS SQL Server (which I use))
- Get familiar with delimited text files and their problems. You can use Excels import feature to import feeds. There you will see fairly quickly what some of the issues are
- never assume that the feed file is okay and that the columns are used for what they are supposed to be used
- Get familiar with FTP (basic FTP that comes with Windows is all you need)
- Get familiar with XMLHTTP to pull data via HTTP
- Get familiar with data validation and data normalization principles and problems
- Learn how to schedule things for automation. I use SQL Server Agent to schedule things, but the Windows scheduler would work for a lot of things too
- Basic XML knowledge would not hurt either
- Get familiar with web services and use them whenever you can.
- Amazon's and eBay's APIs are good for learning and getting experiences with web services
If this is beyond you, have a closer look at the 3rd party services that do all this work for you and provide product data in an easy to use format.
Quote:
Originally Posted by Silencio
Carsten I also found this post on another forum where someone posted a free script that will upload to an MySQL database.
|
That one is for the ShareASale data feeds and not generic, although it could probably be modified to use it for other feeds as well, but you need some PHP knowledge to be able to do that.
Quote:
Originally Posted by Silencio
(If this outside link is not allowed to be posted here please remove it and accept my apologies but as were were discussing this I thought it might be of value...)
|
It is okay, because it is relevant to the discussion
Quote:
Originally Posted by Silencio
If you have uploaded datafeeds directly to your MySQL database today and next week you need to Update your datafeed, how is this accomplished?
|
Two options
You receive in almost every case a full product feed.
1) Delete all records related to the feed in the database and then import it as if it was new. Depending on the feed is the key for the deletion filter usually the merchant ID on the network where you pull the feed from. The problem is that you can't use your own primary key for products in this case, because the key values would change and all your references to the old records would become invalid
2) Check if the product exists in the DB by doing a look-up with a unique identifier for the Merchant+Product combination and do an update if it exists (have a date stamp for "last updated" and update that as well). Insert all records you can't find in the DB. Delete all records where the "last updated" stamp was not updated. This requires more logic for the import script and takes longer to import data.
Hope that helps
Cheers